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In  deze  studie  is  de  rol  van  communicatie  en  coordinatie  bij  teambesluitvorming  onderzocht. 
Hiervoor  is  eerst  een  raamwerk  ontwikkeld  waarin  aan  een  team  gerelateerde  entiteiten  zijn  vervat. 
Dit  raamwerk  beschrijft  dat  een  organisatie  die  haar  doelen  wil  bereiken  te  maken  heeft  met  haar 
omgeving.  De  organisatie  moet  voor  het  realiseren  van  haar  doelen  werk  verzetten  waarvoor  zij 
middelen  als  agenten,  machines  en  gereedschappen  tot  haar  beschikking  heeft.  Wanneer  een  enkele 
agent  niet  over  voldoende  kennis  en  capaciteiten  beschikt  om  het  benodigde  werk  te  verzetten  zal 
een  multi-agent  systeem  worden  gevormd. 

Een  voorbeeld  van  zo’n  multi-agent  systeem  is  een  team.  Om  de  kennis  en  capaciteiten  van  de 
teamleden  te  combineren  moet  het  team  communiceren  en  coordineren,  wat  overhead  vereist  en  een 
bron  van  fouten  kan  zijn.  Problemen  met  team  communicatie  en  coordinatie  kan  tot  desastreuze 
gevolgen  leiden  in  situaties  waar  levens  op  het  spel  staan. 

Om  de  rol  van  communicatie  en  coordinatie  te  onderzoeken  hebben  we  een  kunstmatige  wereld 
ontwikkeld  waarin  verschillende  gebeurtenissen  plaatsvinden.  Deze  wereld  wordt  gerepresenteerd 
door  een  stad  die  bestaat  uit  verscheidende  gebouwen  waarin  zich  burgers  bevinden.  De 
gebeurtenissen  bestaan  uit  een  serie  branden.  Een  brandweerorganisatie  is  opgezet  om  deze  situatie 
het  hoofd  te  bieden.  De  organisatie  moet  haar  beperkte  hoeveelheid  middelen  inzetten  om  haar  doel 
te  bereiken:  het  redden  van  zoveel  mogelijk  burgers. 

Focus  van  het  onderzoek  is  hoe  een  team  samen  te  stellen  en  te  ondersteunen  dat  de  situatie 
observeert  en  beslissingen  neemt  over  het  gebruik  van  de  middelen.  Deze  taken  zijn  in  detail 
geanalyseerd  en  een  normatieve  taakstructuur  is  beschreven  dat  dienst  deed  als  basis  voor  prestatie- 
metingen.  Drie  experimenten  zijn  uitgevoerd.  Ten  eerste  werd  de  veronderstelling  getoetst  dat  de 
experimentele  taak  niet  goed  door  een  enkeling  kan  worden  uitgevoerd.  Ten  tweede  werd 
onderzocht  of  een  team  met  beperkte,  maar  voor  de  taakuitvoering  voldoende,  communicatie-  en 
coordinatiemogelijkheden,  even  goed  presteert  als  een  team  met  onbeperkte  mogelijkheden.  Ten 
derde  werd  onderzocht  wat  de  rol  van  kennisdistributie  is. 

Drie  hoofdconclusies  zijn  getrokken.  Ten  eerste  bleek  dat  cruciale  taakelementen  beter  parallel 
kunnen  worden  uitgevoerd  en  de  taak  dus  door  een  team  moet  worden  uitgevoerd.  Ten  tweede 
kwam  uit  dat  als  communicatie  beperkt  wordt  tot  het  uitwisselen  van  data  dit  de  prestatie 
vermindert.  Ten  derde  lieten  de  resultaten  zien  het  distribueren  van  kennis  over  team  leden  tot  een 
toename  van  parallelle  taakuitvoering  leidt  en  tot  betere  prestatie.  Uit  observaties  kwam  tenslotte 
naar  voren  dat  bij  vrije  communicatie  en  coordinatie  zowel  tijdens  als  tussen  het  uitvoeren  van  taken 
veel  kennis  en  veel  taakinformatie  wordt  uitgewisseld. 

Verder  onderzoek  richt  zich  op  de  ondersteuning  van  team  coordinatie.  Gebruikmakend  van 
hetzelfde  paradigma  worden  twee  condities  vergeleken.  In  beide  condities  zullen  de  teams  beperkt 
kunnen  communiceren  en  coordineren,  maar  in  een  conditie  zal  het  team  actief  worden  ondersteund 
in  de  prioritisering  van  de  taken.  Deze  ondersteuning  zal  bestaan  uit  dynamische  visualisatie  van  de 
taakafhankelijkheden  tussen  degene  die  observeert  en  degene  die  de  middelen  inzet.  De  hypothese  is 
dat  de  uitwisseling  van  coordinatie-informatie  de  prestatie  van  teambesluitvorming  verbetert. 
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SUMMARY 

In  this  study,  the  role  of  communication  and  coordination  in  team  decision  making  in  a 
command  and  control  task  has  been  investigated.  First,  a  framework  has  been  developed  in 
which  all  team-related  entities  are  described.  Next,  we  have  developed  an  artificial  world  in 
which  various  events  take  place.  This  world  is  represented  by  a  city  consisting  of  various 
buildings  in  which  citizens  live.  The  events  are  series  of  fires.  A  fire-fighting  organization  has 
been  set  up  to  deal  with  this  situation.  This  organization  has  to  allocate  a  restricted  amount  of 
resources  to  fulfill  its  goal:  to  rescue  as  many  lives  as  possible. 

Focus  of  research  is  on  how  to  organize  and  support  a  team  that  observes  the  world  and 
allocates  the  resources.  These  tasks  have  been  analysed  and  a  normative  task  strucUire  has 
been  described  as  a  basis  for  performance  measurements.  Two  experiments  have  been  carried 
out.  First,  the  hypothesis  was  tested  that  an  experimental  task  (fire  fighting)  could  not  be 
carried  out  alone.  Second,  it  was  investigated  whether  teams  that  are  restricted  in  their 
communication  and  coordination  possibilities  could  perform  as  well  as  teams  that  can 
communicate  and  coordinate  without  restrictions.  A  third  research  topic  was  the  role  of 
distributing  knowledge  among  the  task  performers. 

The  following  three  main  conclusions  can  be  drawn  from  the  experimental  results.  First,  the 
fire-fighting  task  truly  is  a  team  task  because  for  optimal  task  performance,  crucial  task 
elements  should  be  carried  out  in  parallel.  Second,  communication  restricted  to  data  exchange 
only  leads  to  team  performance  decrease.  Third,  distributing  knowledge  over  team  members 
leads  to  an  increase  of  parallel  task  execution  and  to  better  team  performance.  Finally,  we 
have  observed  that  when  team  members  can  communicate  without  restrictions,  much 
knowledge  and  information  about  the  task  is  exchanged,  both  during  and  between  the 
execution  of  tasks. 
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SAMENVATTING 

In  deze  studie  is  de  rol  van  communicatie  en  coordinatie  bij  teambesluitvorming  onderzocht. 
Hiervoor  is  eerst  een  raamwerk  ontwikkeld  waarin  aan  een  team  gerelateerde  entiteiten  zijn 
vervat.  Vervolgens  hebben  we  een  kunstmatige  wereld  ontwikkeld  waarin  verschillende 
gebeurtenissen  plaatsvinden.  Deze  wereld  wordt  gerepresenteerd  door  een  stad  die  bestaat  uit 
verscheidende  gebouwen  waarin  zich  burgers  bevinden.  De  gebeurtenissen  bestaan  uit  een 
serie  branden.  Een  brandweerorganisatie  is  opgezet  om  deze  situatie  bet  hoofd  te  bieden.  De 
organisatie  moet  haar  beperkte  hoeveelheid  middelen  inzetten  om  haar  doel  te  bereiken:  bet 
redden  van  zoveel  mogelijk  burgers. 

Focus  van  bet  onderzoek  is  boe  een  team  samen  te  stellen  en  te  ondersteunen  dat  de  situatie 
observeert  en  beslissingen  neemt  over  bet  gebruik  van  de  middelen.  Deze  taken  zijn 
geanalyseerd  en  een  normatieve  taakstructuur  is  bescbreven  die  dienst  deed  als  basis  voor 
prestatie-metingen.  Drie  experimenten  zijn  uitgevoerd.  Ten  eerste  werd  de  veronderstelling 
getoetst  dat  de  experimentele  taak  niet  goed  door  een  enkeling  kan  worden  uitgevoerd.  Ten 
tweede  werd  onderzocbt  of  een  team  met  beperkte,  maar  voor  de  taakuitvoering  voldoende, 
communicatie-  en  coordinatiemogelijkbeden,  even  goed  presteert  als  een  team  met  onbeperkte 
mogelijkbeden.  Ten  derde  werd  onderzocbt  wat  de  rol  van  kennisdistributie  is. 

Drie  boofdconclusies  zijn  getrokken.  Ten  eerste  bleek  dat  cruciale  taakelementen  beter 
parallel  kunnen  worden  uitgevoerd  en  de  taak  dus  door  een  team  moet  worden  uitgevoerd. 
Ten  tweede  kwam  naar  voren  dat  als  communicatie  beperkt  wordt  tot  bet  uitwisselen  van  data 
dit  de  prestatie  vermindert.  Ten  derde  lieten  de  resultaten  zien  dat  bet  distribueren  van  kennis 
over  team  leden  tot  een  toename  van  parallelle  taakuitvoering  leidt  en  tot  betere  prestatie.  Uit 
observaties  kwam  tenslotte  naar  voren  dat  bij  vrije  communicatie  en  coordinatie  zowel  tijdens 
als  tussen  bet  uitvoeren  van  taken  veel  kennis  en  veel  taakinformatie  wordt  uitgewisseld. 
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1  INTRODUCTION 
1.1  Bacl^ound 

In  most  organisations,  many  jobs  cannot  be  done  alone,  but  have  to  be  carried  out  in  a  team. 
The  members  of  a  team  may  differ  in  background,  expertise  and  abilities.  Many  teams  have 
to  perform  their  tasks  in  complex  work  settings.  Example  teams  are  command  &  control 
teams,  fire  fighters,  cockpit  crews,  and  medical  teams.  The  work  setting  of  these  teams  may 
be  characterized  by  time  pressure,  heavy  work  load,  ambiguous  information,  and  a  constantly 
changing  environment.  Dysfunctioning  of  one  or  more  members,  or  miscommunication 
among  the  team  members,  can  lead  to  disastrous  consequences.  A  major  challenge  is  to  find 
out  how  a  team  should  cooperate  to  reach  optimal  performance  under  demanding 
circumstances.  In  particular,  we  are  interested  in  determining  the  optimal  conditions  for  a 
team  to  make  decisions  in  critical  situations. 

Team  decision  making  has  received  a  great  deal  of  attention  in  recent  years  (see,  for  example, 
Cannon-Bowers,  Salas  &  Converse,  1993;  Converse,  Cannon-Bowers  &  Salas,  1991; 
Orasanu  &  Salas,  1993;  Salas,  Bowers  &  Cannon-Bowers,  1995).  This  interest  of  researchers 
is  partly  due  to  recent  incidents  which  have  demonstrated  the  importance  of  improving  the 
performance  of  teams.  For  example,  a  commercial  Iranian  airliner  was  misidentified  and  shot 
down  by  the  crew  aboard  the  USS  Vincennes.  The  crew  misinterpreted  the  data  and  was 
consequently  convinced  that  they  were  dealing  with  an  attacking  military  aircraft.  This 
example  shows  how  poor  team  decision  making  can  affect  the  lives  of  people.  The  research  in 
this  report  was  motivated  by  our  interest  in  the  factors  involving  team  decision  making.  This 
in  order  to  formulate  recommendations  for  enhancing  team  performance  and  to  prevent  future 
accidents  as  a  result  of  poor  team  decision  making. 

Before  providing  an  overview  of  the  literature  on  team  decision  making,  we  will  first  explain 
what  we  mean  when  we  speak  of  a  team.  Dyer  (1984)  defined  a  team  as  follows: 

“A  team  consists  of  (a)  at  least  two  people,  who  are  (b)  working  together  towards  a 
common  goal/objective/mission,  where  (c)  each  person  has  been  assigned  specific  roles 
or  functions  to  perform,  and  where  (d)  completion  of  the  mission  requires  some 
dependency  among  the  group  members”  (p.286) 

According  to  Orasanu  and  Salas  (1993),  the  dependency  among  the  members  consists  of 
information,  knowledge,  and  means  for  reaching  their  common  goal.  Throughout  this  work, 
we  will  use  the  definition  by  Dyer  and  the  further  explication  by  Orasanu  and  Salas. 


1.2  Goal 

This  work  has  multiple  goals.  First,  we  will  try  to  define  what  constitutes  a  team  by  reviewing 
the  literature  on  team  decision  making  and  by  developing  a  framework  for  knowledge  and 
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information  used  within  a  team.  Second,  we  report  on  empirical  research  on  team  decision 
making  that  has  been  carried  out  in  our  institute.  Third,  we  will  formulate  future  research 
directions  and  a  new  research  paradigm  to  improve  our  understanding  of  team  decision 
making  further.  Our  ultimate  goal  is  to  predict  team  performance  as  reliably  as  possible. 


1.3  Research  question 

A  team  combines  the  knowledge  and  capacities  of  its  members  to  deal  with  situations  a  single 
agent  can  not  handle  alone.  A  drawback  of  this  is  that  team  members  have  to  communicate 
and  coordinate  their  activities,  which  takes  time,  requires  effort,  and  can  be  a  source  of 
errors.  This  work  focuses  on  these  two  elements  of  team  decision  making.  We  are  especially 
interested  in  the  factors  that  play  a  role  in  team  performance  when  team  members  have  to 
communicate  and  coordinate  under  critical  circumstances. 

That  effective  communication  and  coordination  can  be  vital  in  some  situations  was  illustrated 
by  the  recent  disaster  with  the  Hercules  military  transport  aircraft  of  the  Belgian  air  force  (see 
Duin  &  Rosenthal,  1996).  During  the  landing  at  a  Dutch  airport,  the  Hercules  got  in  trouble 
and  crashed.  Serious  problems  with  communication  and  coordination  arose  when  the 
emergency  services  provided  their  assistance.  Normally,  a  load  carrier  plane  like  the  Hercules 
contains  a  crew  of  four  individuals.  This  time,  however,  the  plane  transported  a  brass  band  of 
the  Dutch  Royal  Army  consisting  of  37  people.  Although  the  air-traffic  controller  knew  that 
people  were  transported,  this  information  was  not  distributed  or  was  misunderstood  by  the 
emergency  services.  Consequently,  the  emergency  services  allocated  less  resources  than 
necessary  and  were  interested  in  the  cockpit  instead  of  the  rear  end  of  plane  where  the 
transported  people  were  seated.  When  the  kind  of  load  would  have  been  known  by  the 
emergency  services  from  the  begirming,  a  different  strategy  would  have  been  adopted  and 
probably  more  lives  could  have  been  saved. 

Although  accurate  communication  is  of  great  importance,  it  is  just  a  part  of  the  picture.  In 
rapidly  changing  situations  teams  may  have  to  cope  with  ambiguous  information  or 
information  that  is  out  of  date.  Moreover,  in  non-routine  and  new  situations  it  even  may  be 
not  clear  which  information  can  be  perceived  as  useful.  On  top  of  this,  communication  does 
not  merely  serves  a  function  of  distributing  accurate  data  information  (i.e.,  information 
needed  to  complete  the  task  such  as  the  number  of  casualties  in  the  Hercules  incident)  but  also 
a  function  of  coordination  and  correction.  An  important  function  that  is  served  by 
communication  is  the  possibility  to  monitor  and  correct  ongoing  activities.  This  is  known  as 
team-self  correction  and  can  be  seen  as  a  specific  extension  on  the  traditional  feedback 
concept.  According  to  Blickensderfer,  Cannon-Bowers  and  Salas  (in  press)  “it  is  contented 
that  intra-team  feedback  (i.e.,  members  giving  each  other  feedback  regarding  how  they  are 
performing  the  task  and  their  underlying  expectations  for  performance),  can  help  to  foster 
correct  expectations  and  intentions  among  team  members  and  also  facilitate  modifications  of 
incorrect  expectation  and  intentions”.  When  in  the  Hercules  case  the  air-traffic  controller 
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would  have  had  infonnation  about  how  the  emergency  services  coordinated  their  actions,  he 
could  have  inferred  that  they  had  chosen  the  wrong  strategy  (i.e.,  to  start  at  the  cockpit  in 
stead  of  the  rear  end;  and  using  a  restricted  amount  of  resources). 

Rouse,  Cannon-Bowers  and  Salas  (1992)  suggest  that  effective  coordination  and 
communication  within  teams  depends  on  correct  expectations  and  interpretations  of  what 
others  will  do.  In  other  words,  in  a  well  coordinated  team,  members  hold  accurate 
expectations  of  each  others  tasks  and  activities.  Furthermore,  team  members  also  have  an 
accurate  interpretation  of  this.  Holding  the  right  expectations  and  interpretations  gives  team 
members  the  opportunity  to  anticipate  each  others  activities  and  provide  essential  information 
when  necessary.  This  results  in  what  Serfaty,  Entin  and  Volpe  (1993)  call  “implicit 
coordination”.  In  sum,  self  monitoring  of  teams  foster  the  development  of  adequate 
expectations  of  each  others  tasks  and  activities.  On  the  basis  of  these  expectations,  teams  can 
coordinate  and  communicate  efficiently.  Both  Blickensderfer  et  al.  as  well  as  Rouse  et  al. 
elaborate  their  view  of  developing  accurate  expectations  to  the  shared  mental  model  concept. 
They  suggest  that  this  concept  can  be  used  as  a  framework  that  gives  an  explanation  how 
people  develop  compatible  expectations  and  interpretations  of  each  other  and  the  task. 

When  members  of  a  team  share  knowledge  from  the  outside  world  (whether  or  not  as  part  of 
a  shared  mental  model),  a  team  as  a  whole  may  be  less  susceptible  to  inaccuracies.  Team 
members  can  fill  in  each  others  deficiencies  and  correct  their  shortcomings.  Also  team 
members  can  substitute  for  each  others  tasks.  This  way  activities  of  individual  team  members 
may  be  smoothly  coordinated.  In  particular,  shared  knowledge  can  be  useful  in  situations 
where  communication  among  team  members  is  not  optimal.  Redundancy  in  knowledge  among 
different  agents  within  a  team  can  compensate  for  such  situations  because  shared  knowledge 
does  not  need  to  be  communicated.  This  has  been  well  illustrated  with  the  Hercules  disaster. 
If  the  knowledge  about  the  number  of  passengers  was  shared  among  all  participants,  problems 
with  the  assistance  probably  could  have  been  prevented. 

Although  shared  knowledge  can  partly  compensate  for  non-optimal  communication,  this  may 
not  be  realizable  in  all  situations.  This  is,  for  example,  when  members  of  a  team  have  at  their 
disposal  highly  specialized  knowledge  that  takes  years  of  training  to  acquire.  Besides  that, 
shared  knowledge  cannot  be  tested  on  its  correctness  when  team  members  do  not  monitor 
each  others  knowledge.  Especially  in  rapidly  changing  situations  shared  knowledge  may  soon 
be  out  of  date.  When  people  monitor  this  disagreement  between  the  situation  and  their 
knowledge  they  have  the  oppormnity  to  correct  and  adjust  their  knowledge.  As  in  the  case 
with  the  Hercules  it  is  not  very  likely  that  all  participants  knew  the  number  of  passengers 
before  the  plane  crashed.  In  other  words,  altogether  sharing  certain  knowledge  among  team 
members  may  enhance  coordination  (and  as  a  result  performance  levels),  it  is  not  applicable 
in  every  situation.  Therefore  we  expect  that,  apart  from  shared  knowledge  that  contributes  to 
the  overall  team  performance,  team  performance  will  be  further  improved  when  team 
members  have  the  possibility  of  self-monitoring  and  self-correction. 
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According  to  Zsambok  (1997)  “many  researchers  agree  that  good  teams  monitor  their 
performance  and  self-correct;  offer  feedback;  maintain  awareness  of  rules  and  functions  and 
take  action  consistent  with  that  knowledge;  adapt  to  changes  in  the  task  or  the  team; 
communicate  effectively;  converge  on  a  shared  understanding  of  their  situation  and  course  of 
action;  anticipate  each  other’s  actions  or  needs;  and  coordinate  their  actions”  (p.ll2).  The 
problem  is,  however,  that  these  factors  are  intuitively  appealing  but  need  to  be  empirically 
tested.  The  research  described  in  this  report  provides  an  experimental  environment  which  can 
be  used  to  investigate  systematically  certain  factors  of  communication  and  coordination.  Our 
main  objective  is  to  compare  teams  with  full  possibilities  for  communication  with  fewer 
possibilities  for  communication.  In  both  situations,  however,  teams  have  the  possibility  to 
communicate  the  necessary  information  for  an  optimal  performance. 

Before  we  address  the  factors  concerning  the  communication  in  teams,  we  would  like  to 
demonstrate  that  some  tasks  are  so  demanding  that  more  than  one  agent  is  needed  to  perform 
the  task.  Primarily,  this  serves  as  a  verification  that  the  experimental  task  used  really  is  a 
team  task.  Therefore  the  first  question  we  want  to  investigate  is: 

Can  we  show  the  necessity  of  petforming  certain  tasks  by  more  than  one  agent? 
Second,  we  want  to  demonstrate  that  when  team  members  are  restricted  in  their  possibilities 
to  communicate  (i.e.,  no  communication  concerning  task  and  domain  knowledge,  thus,  no 
opportunity  for  self-monitoring  or  coordination)  performance  declines.  This,  in  spite  of  the 
fact  that  there  are  sufficient  possibilities  for  communication  in  order  to  perform  the  task  well. 
The  following  research  question  will  be  investigated: 

Will  unrestricted  communication  contribute  to  a  better  performance  ? 

Third,  we  want  to  demonstrate  that,  even  when  team  members  are  restricted  in  their 
possibilities  to  communicate,  shared  knowledge  can  partly  compensate  for  this  omission.  The 
following  research  question  will  be  investigated: 

Will  the  possession  of  shared  knowledge  among  team  members  contribute  to  a  better 
performance? 

In  the  next  section,  we  will  analyse  what  types  of  knowledge  and  information  a  team  has  to 
deal  with. 


2  TEAM  KNOWLEDGE  AND  INFORMATION  FRAMEWORK 

What  types  of  knowledge  and  information  are  used  by  a  team  and  how  are  they  processed? 
This  is  the  main  question  that  guides  our  review  of  the  literature  on  team  decision  making 
presented  in  this  section. 

An  often  used  concept  in  describing  team  functioning  is  Shared  Mental  Model  (Orasanu, 
1993;  Orasanu  &  Salas,  1993).  This  concept  is  based  on  the  mental  model  concept.  By  this  is 
meant  “[...]  the  mechanisms  whereby  humans  are  able  to  generate  descriptions  of  system 
purpose  and  form,  explanations  of  system  functioning  and  observed  system  states,  and 
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predictions  of  future  system  states”  (Rouse  &  Morris,  1986,  p.351).  Agents  use  mental 
models  to  describe  the  purpose  and  form  of  a  system,  to  explain  a  system  in  terms  of  its 
functions  and  its  state  (what  it  is  doing),  and  predicting  a  future  state. 

A  shared  mental  model  represents  the  intersection  of  the  mental  models  of  at  least  two 
individual  agents.  Orasanu  and  Salas  (1993)  include  shared  understanding  of  the  problem, 
goals,  information  cues,  strategies,  and  member  roles  in  these  models.  They  argue  that  this 
shared  mental  model  is  important  for  team  decision  making  in  order  to  enable  team  members 
to  cooperate  optimally.  By  using  a  shared  mental  model,  a  team  would  know  how  to 
coordinate  its  actions  and  what  to  communicate  among  the  members  of  the  team. 

A  related  concept  to  shared  mental  model  is  team  mind  (Orasanu  &  Salas,  1993),  which  refers 
to  shared  information  processing  capabilities,  especially  memory  encoding,  storage  and 
retrieval.  Each  team  member  knows  what  problem  information  and  what  expertise  other 
members  have.  In  this  way,  the  individual  knowledge  and  information  storage  capacity  is 
expanded.  According  to  Orasanu  &  Salas,  shared  mental  models  and  team  mind  should  be 
regarded  as  conceptual  frameworks  for  research  on  teams  making  decisions  rather  than  fully 
developed  theories. 

In  addressing  the  mental  models  team  members  may  have  we  will  use  five  levels:  the  situation 
level  (i.e.,  factors  in  the  external  environment  of  the  team,  such  as  the  dynamics),  the 
organisation  level  (e.g.,  work  place,  distance  and  time  separation),  the  task  level  (the  tasks 
that  should  be  performed  to  realize  organizational  goals),  the  agent  level  (the  performers  of 
the  tasks,  who  are  limited  in  knowledge  and  capacity),  the  team  level  (the  set  of  agents  that 
carry  out  the  tasks  together,  using  knowledge  about  each  others  tasks,  perform  coordination 
and  communication,  etc.).  We  will  use  these  layers  to  structure  the  overview  of  the  literature. 


2.1  Organizations  are  dealing  with  a  situation 

What  is  meant  by  situation.  Organizations  function  within  a  context.  For  ship  bridge 
personnel,  this  context  consists  of  the  environment  (such  as  weather  and  current),  the  traffic, 
and  the  hydrography.  For  commercial  organizations,  elements  of  the  situation  are  the  market, 
the  macro-economic  simation,  the  competition,  and  technological  developments.  The  external 
situation  should  be  separated  from  the  internal  situation  of  the  organization  (such  as  resources, 
workplaces,  ete.). 

Non-determinism  of  the  situation.  The  impact  an  organization  can  have  on  a  situation  can  only 
be  partial.  Many  actors  and  factors  have  an  influence:  competitors  in  the  market,  customer 
trends,  adversaries.  In  other  words,  the  influence  on  the  situation  is  intrinsically  unpredictable 
(Orasanu  &  Connolly,  1993).  However,  the  impact  of  an  organization  on  the  situation  can  be 
estimated. 


12 


A  difference  exists  between  a  situation  and  a  situation  description.  There  is  a  difference 
between  the  real  external  situation  and  how  it  is  known  to  the  organization.  The  external 
situation  can  only  be  described  partially  since  the  amount  of  information  is  too  large  to  be 
captured  or  is  impossible  to  obtain  otherwise.  And  as  with  every  description,  problems  can 
arise  with  the  correcmess,  completeness,  consistency  of  the  description.  One  of  the  main  tasks 
of  an  organization  is  keep  the  description  as  accurate  as  possible.  But  still,  the  situation  is 
mostly  described  in  terms  of  estimates,  probabilities,  etc.  Describing  explicitly  what  is 
unknown  may  be  equally  important.  Rasmussen  (1993)  points  out  that  decisions  within  an 
organization  often  should  be  based  on  partial  external  information.  People  are  able  to 
compensate  and  correct  incomplete,  unreliable  and  ambiguous  information  by  means  of  a 
coherent  mental  model  people  have  of  the  situation.  According  to  Orasanu  and  Connolly 
(1993),  available  information  about  the  situation  may  even  be  distorted  intentionally,  for 
example  to  mislead  an  adversary.  Therefore,  each  part  of  a  situation  description  that  might 
have  originated  from  an  adversary  should  be  regarded  as  suspect. 

Dynamic  situations,  hectic  situations.  Another  important  characteristic  of  a  situation, 
identified  by  many  authors,  is  the  dynamics  of  a  situation  (e.g..  Rouse  et  al.,  1992;  Orasanu 
&  Connolly,  1993).  A  situation  can  change  rapidly,  and  so  is  the  impact  for  the  organization. 
The  organization  should  be  able  to  adapt  quickly  to  a  new  situation,  meaning  that 
organizational  goals,  priorities  and  procedures  may  need  revision.  For  example,  the  manning 
on  the  ship  bridge  should  adapt  their  travel  plan  according  to  changing  weather  conditions. 
The  time  pressure  under  which  a  team  may  have  to  function  may  influence  the  functioning  of 
a  team  importantly.  The  dynamic  character  of  the  situation  gave  rise  to  a  lot  of  research,  such 
as  the  TADMUS  program  (Tactical  Decision  Making  Under  Stress)  (e.g..  Rouse  et  al.,  1992; 
Zachary,  Zaklad,  Hicinbothom  &  Ryder,  1993).  This  program  was  initiated  after  the 
command  &  control  team  of  the  USS  Vincennes  mistook  an  Iranian  airliner  for  an  aversive 
military  plane.  When  the  plane  was  within  range  to  attack  the  ship,  the  team  decided,  within  a 
time  period  of  only  minutes,  to  launch  a  missile  in  order  to  defend  itself. 

The  impact  of  a  changing  situation.  Not  only  the  lack  of  time  to  make  a  decision  induces 
stress,  also  the  impact  that  a  wrong  decision  may  have.  A  mistake  may  lead  to  the  fall  of  an 
enterprise,  the  discharge  of  the  responsible  people,  or  the  uimecessary  death  of  human  beings, 
as  has  been  the  case  in  the  Vincennes  accident. 

Naturalistic  decision  making.  Aspects  such  as  the  uncertainty  and  dynamics  of  the  situation, 
and  the  impact  of  a  changing  situation  for  the  organization  and  their  individuals,  are  some  of 
the  topics  in  a  relatively  new  research  paradigm,  called  naturalistic  decision  making  (NDM, 
see  e.g.,  Klein  et  al.,  1993).  The  objective  of  NDM  research  is  finding  out  how  knowledge, 
experience,  and  skills  are  actually  used  in  the  decision  making  process  in  uncertain  and 
demanding  situations,  as  opposed  to  a  normative  approach  of  decision  making. 

The  new  paradigm  brings  together  two  problems.  First,  it  is  recognized  that  decision  makers 
in  practice  do  not  apply  probability  theory  and  utility  theory,  in  which  options  are  generated 
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and  weighted,  and  the  best  option  is  selected.  Probability  theory  and  utility  theory  are 
perfectly  applicable  in  situations  such  as  with  gambles,  when  the  situation  is  a  standard  one 
and  sufficient  time  is  available  for  the  necessary  calculations.  The  second  problem  with 
probability  theory  and  utility  theory  is  that  the  conditions  may  not  change.  Deciding,  for 
example,  to  bet  some  money  in  a  game  such  as  roulette  does  not  change  the  probability  or 
utility  of  the  outcome.  However,  it  is  often  the  objective  to  change  the  current  situation.  In 
that  case,  experience  from  previous  situations,  described  by  probabilities,  is  not  useful  since 
this  experience  is  not  applicable  for  the  new  situation.  In  addition,  under  new  circumstances, 
other  factors  not  known  yet  may  have  become  important.  An  example  of  a  changing  situation 
in  gambling  is  when  the  bank  explodes:  probability  and  utility  theory  do  not  work  any  more 
in  that  case. 

Instead  of  calculating  with  probabilities,  decision  makers  in  actual  practice  recognize  a 
situation  as  a  familiar  one,  and  act  upon  it  in  a  standard  way.  Sometimes,  when  the  situation 
is  unfamiliar,  they  try  to  reason  away  the  differences,  and  again  use  their  standard  repertoire. 
In  other  cases,  when  they  are  not  convinced  that  their  standard  repertoire  of  action  is 
appropriate,  they  can  mentally  simulate  possible  scenarios,  in  order  to  test  and  adapt  the 
plaimed  course  of  action  (Klein,  1997). 

How  to  describe  the  dynamics.  Basically,  the  dynamics  of  a  situation  can  be  described  in  two 
different  manners:  process-based  and  event-based.  In  a  process-based  description,  the 
situation  is  described  after  a  fixed  period  of  time.  How  much  time  will  elapse  between  the 
points  in  time  may  depend  on  the  effort  needed  to  refresh  the  description  and  the  possible 
situational  change  within  that  period  of  time.  For  describing  process-based  dynamics,  only  the 
notion  of  time  is  important  (besides  knowledge  of  how  to  recognize  and  describe  the  objects 
in  the  world). 

In  an  event-based  description,  only  qualitatively  different  situations  are  described.  In  this 
description  a  notion  of  what  is  “qualitatively  different”  is  essential.  This  is  determined  by  the 
activities,  or  tasks,  that  can  be  carried  out  to  handle  the  event.  As  we  will  see  below,  these 
tasks  are  important  for  describing  team  functioning. 


2.2  The  organization 

In  contrast  with  the  context  of  an  organization,  the  organization  itself  can  be  controlled 
directly.  An  organization  consists  of  resources  that  can  be  allocated  and  managed.  These 
resources  are  the  manpower  of  the  organization,  and  means  such  as  money  and  machines. 
Knowledge  can  be  regarded  as  an  organizational  asset  as  well  (see  Post  et  al.,  1997). 

Goal  and  mission.  The  resources  are  used  to  reach  a  goal,  or  to  fulfill  a  mission.  A  goal  is 
usually  formulated  as  a  future  simation  the  organization  is  aiming  for;  a  mission  as  the 
process  for  reaching  this  goal.  The  mission  of  an  organization  is  accomplished  by  means  of 
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organizational  functions.  For  a  commercial  organization,  example  functions  are  sales, 
production  and  research  &  development.  For  a  ship,  the  functions  are,  for  example, 
navigation,  propulsion  and  lookout. 

The  future  situation  should  be  described  in  order  to  be  known  for  the  members  of  the 
organization.  As  with  the  description  of  a  situation,  the  description  can  be  incorrect, 
incomplete  and  inconsistent.  Further,  a  goal  is  not  static.  It  can  change  due  to  different 
external  circumstances  and  the  available  resources  (Orasanu  &  Connolly,  1993).  For  this 
reason,  and  for  dealing  with  unforeseen  situations,  organizational  flexibility  is  essential 
(Rouse  et  al.,  1992). 

Teams  within  an  organization  may  function  at  a  distance  and  asynchronously.  Resources  such 
as  humans  and  machines  consume  time  and  space.  Therefore,  organizations  have  to  structure 
their  resources  in  work  processes  and  workplaces.  A  design  option  for  this  structuring  is  to 
separate  human  resources  in  time  and  space.  Working  at  a  distance  and  asynchronously  may 
have  an  impact  on  the  functioning  of  the  organization.  Two  decades  ago,  Chapanis,  Parrish, 
Ochsman  and  Weeks  (1977)  investigated  working  at  a  distance  via  teletypewriters.  Due  to  the 
technological  developments  on  multimedia  and  telecommunications  technology  since,  much 
research  is  carried  out  on  computer  communication,  shared  workspaces  such  as  supported  by 
conferencing  systems,  etc.  (e.g.,  Fussell  &  Benimoff,  1995). 

Organizational  resources  have  to  be  compiled.  The  management  and  allocation  of  resources  is 
one  of  the  criteria  that  define  a  team  (Salas,  Dickenson,  Converse  &  Tannenbaum,  1992). 
Adaptation  of  an  organization  to  a  changing  situation  (or  changing  goal  situation)  consists  of 
the  allocation  of  resources.  When  the  organization  is  not  able  to  cope  with  the  demands  of  a 
situation,  resources  should  be  added  (and  withdrawn  from  a  place  in  the  organization  where  it 
is  less  needed).  These  resources  may  be  human  as  well  as  machine  power.  The  allocation  of 
manpower  is  constrained,  however.  The  number  of  members  a  team  should  consist  of  has  an 
optimal  size:  when  the  number  is  too  small,  problems  with  backups  may  arise;  when  the 
number  is  too  large,  the  overhead  becomes  a  problem  (see  for  example  Mintzberg,  1983,  for 
an  account  on  organizational  structures).  In  addition,  the  members  of  an  organization  need  to 
get  used  to  each  other’s  ways,  which  takes  time  (Van  Delft  &  Schuffel,  1995).  For  dynamic 
allocation,  this  start  up  time  may  take  too  long.  Therefore,  a  permanent  team  may  be  installed 
which  is  oversized  to  be  able  to  handle  an  incidental  demanding  situation. 


2.3  Organizational  goals  are  realized  by  performing  tasks 

Caimon-Bowers  et  al.  (1993)  explicitly  incorporate  tasks  in  their  definition  of  shared  mental 
models:  “shared  mental  models  are  defined  as  knowledge  structures  held  by  members  of  a 
team  that  enable  them  to  form  accurate  explanations  and  expectations  for  the  task,  and,  in 
turn,  to  coordinate  their  actions  and  adapt  their  behaviours  to  the  demands  of  the  task  and 
other  team  members”  (p.228). 
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A  model  of  team  tasks  is  an  essential  part  of  a  shared  mental  model.  We  define  a  task  as 
“activities  consuming  time  and  effort  in  order  to  achieve  a  goal”.  The  time  element  refers  to 
the  process  in  which  tasks  are  realized;  effort  refers  to  the  need  for  resources  that  provide  the 
effort  to  carry  out  tasks.  Three  types  of  tasks  can  be  distinguished:  sensor  tasks,  motor  tasks, 
and  cognitive  tasks. 

Tasks  are  structured  in  processes.  They  can  start  and  stop  at  any  moment;  run  sequentially  in 
a  particular  order  or  in  parallel;  they  can  be  repeated,  interrupted,  etc.  The  structuring 
depends  on  the  dependency  between  tasks;  a  particular  task  may  not  start  before  another  task 
is  completed. 

The  difference  between  function  and  task  is  the  activity  component.  Functions  are  not  defined 
in  terms  of  activities,  and  therefore  require  no  time  and  effort.  Different  types  of  tasks  exist. 
Following  Chandrasekaran  (1987),  a  number  of  authors  assume  that  complex  tasks  can  be 
decomposed  into  a  number  of  generic  tasks.  For  military  domains.  Van  Delft  and  Schuffel 
(1995)  propose  four  main  tasks:  situation  awareness,  threat  assessment,  planning/decision 
making  and  direction  &  control.  According  to  Adams  (1995)  the  set  of  generic  tasks  are 
coordination,  situation  assessment,  plan  generation,  and  plan  implementation.  Van  Heijst, 
Lanzola,  Schreiber  and  Stefanelli  (1994)  use  three  similar  basic  tasks  for  the  medical  domain: 
diagnosis,  therapy  planning  and  patient  monitoring.  Hollingshead  and  McGrath  (1995) 
distinguish  the  domain-independent  tasks  generate  (creativity  tasks,  planning  tasks),  choose 
(intellective  tasks,  decision  making  tasks),  negotiate  (cognitive  conflict  tasks,  mixed  motive 
tasks),  and  execute  (contests/competitive  tasks,  performance/psychomotor  tasks). 


2.4  Tasks  are  carried  out  by  agents 

Capacities.  Tasks  that  realize  functions  are  carried  out  by  agents.  Agents  can  be  humans, 
machines,  or  software.  For  performing  tasks,  agents  need  capacities.  The  capacities  of  an 
individual  agent  refer  to  sensor,  motor  and  cognitive  capabilities  (i.e.,  capacities  to  reason 
and  conununicate).  Wickens  (1992)  provides  a  model  of  human  information  processes  that  can 
serve  to  investigate  the  limitations  of  these  capacities.  In  this  model,  a  (human)  agent  carries 
out  visual,  auditory  and  kinesthetic  sensory  processing  to  handle  stimuli.  A  short-term  sensory 
store  for  each  modality  represents  the  details  of  the  stimulus  for  a  short  period  of  time  after  it 
has  occurred  physically.  For  visual  information,  this  period  is  less  then  a  second,  for  the  other 
two  modalities  two  to  eight  seconds.  No  conscious  attention  is  required  for  short  term  sensory 
store. 

The  next  step  is  encoding  the  stimulus  permanently  in  long-term  memory  as  a  single 
perceptual  category  after  it  has  been  perceived  or  recognized  (i.e.,  has  been  linked  to 
previously  learned  perceptual  categories  already  stored  in  long-term  memory).  Different 
levels  of  categorization  can  be  distinguished.  Detection  is  a  task  at  the  least  complex  level. 
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Recognition,  identification  and  categorization  are  more  complex  tasks.  When  more  categorical 
dimensions  are  involved,  pattern  recognition  takes  place. 

After  the  stimulus  is  encoded,  the  agent  has  to  decide  what  action  should  follow.  One 
possibility  is  to  select  a  response.  Deciding  what  to  do  can  be  made  after  some  considerations 
but  also  immediately.  Another  possibility  is  to  delay  an  action  by  storing  the  information 
temporarily  into  the  working  memory.  To  retain  this  information  in  the  working  memory  it 
should  be  actively  rehearsed.  A  second  option  is  to  learn  it  by  storing  it  in  long  term  memory. 

After  a  response  is  selected  it  will  be  executed.  The  response  uses  a  particular  plan  to  perform 
the  action.  The  execution  of  the  plan  is  monitored  in  a  closed-loop  feedback  structure. 

When  the  resources  are  still  insufficient  to  carry  out  a  task,  one  option  is  left:  forming  a  team 
of  multiple  agents.  Within  this  team,  agents  have  a  position  (e.g.,  a  position  in  a  hierarchy) 
which  gives  them  responsibility,  and  they  must  act  within  certain  constraints.  These 
constraints  can  be  divided  in  permissions  (formal  agreements  within  an  organization  on  the 
performance  of  particular  tasks),  norms  (ethical  considerations  in  performing  a  task),  and 
preferences  (about  how  to  carry  out  a  (combination  of)  particular  tasks)  (Wasm  &  Gala, 
1993). 

Stress.  Information  processing  capacities  are  influenced  by  stress  factors,  such  as  noise, 
anxiety  and  incentive,  fatigue  and  sleep  loss  etc.  Effects  of  stress  may  be  attentional 
narrowing,  working  memory  loss,  shifts  in  the  relation  of  speed  versus  risk,  and  decision 
making  performance  decreases  (Wickens,  1992). 

Knowledge.  (Human)  agents  store  three  types  of  information  in  their  long-term  memory: 
skills,  rules,  and  knowledge  (Rasmussen,  1983).  Skills  refer  to  automated  and  highly 
integrated  patterns  of  behaviour.  Rules  are  condition-action  relations  to  handle  previously 
experienced  situations.  Knowledge  refers  to  models  one  has  of  (a  part  of)  the  world.  By  using 
this  knowledge  problems  not  encountered  before  can  be  solved.  A  number  of  authors 
distinguish  two  types  of  knowledge:  domain  knowledge  (i.e.,  facts  within  a  particular  domain) 
and  problem  solving  knowledge  (knowledge  about  how  to  make  inferences  given  the  problem 
description  and  how  to  combine  these  inferences  to  solve  the  problem)  (e.g.,  Clancey,  1985; 
Wielinga  &  Breuker,  1986;  Van  Heijst  et  al.,  1994). 

Learning.  Skills,  rules  and  knowledge  are  acquired  through  the  process  of  learning. 
Knowledge  is  either  passed  through  by  others  or  is  acquired  by  experience.  Becoming  skilful 
or  an  expert  in  a  certain  domain  is  time-consuming.  Therefore,  skills  and  expertise  are 
distributed  over  the  members  of  a  team  rather  than  similar  for  each  member.  Further, 
knowledge,  once  acquired,  is  not  static  but  changes  through  task  execution.  This  process 
needs  feedback  and  knowledge  of  results. 
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2.5  Agents  can  form  a  team 

As  has  been  pointed  out  above,  the  knowledge  and  capacities  of  an  agent  are  limited.  For 
certain  tasks,  these  limitations  prevent  the  agent  to  perform  the  job.  One  of  the  possibilities  to 
extend  the  knowledge  and  capacities  by  carrying  out  the  job  with  more  than  one  agent.  Van 
de  Velde  &  Perram  (1996)  define  multi-agent  systems  as  “[...]  distributed  systems  that  are 
designed  as  a  collection  of  interacting  autonomous  agents,  each  having  their  own  capacities 
and  goals  that  are  related  to  a  common  environment.  ”  In  the  definition  of  team  given  by  Dyer 
(1984),  cited  in  the  introduction,  three  factors  are  important  for  distinguishing  teams  from 
other  multi-agent  systems:  cooperation  (to  realize  a  shared  mission),  interdependency  (the 
mission  cannot  be  fulfilled  without  the  other  members),  and  heterogeneity  (the  members  differ 
with  respect  to  expertise,  roles  and  responsibility).  Three  typical  characteristics  of  teams  can 
be  added.  First,  the  interdependency  is  often  caused  by  the  need  for  real-time  problem 
solving;  a  single  agent  with  limited  knowledge  and  capacities  cannot  just  take  more  time  to 
finish  his  or  her  job,  because  there  is  a  deadline.  Moreover,  when  real-time  problem-solving 
should  be  carried  out  continuously,  shifts  are  necessary  to  refresh  a  team.  Second,  teams  are 
limited  in  size.  Otherwise,  the  overhead  in  communication  and  coordination  decreases  team 
performance.  Third,  teams  are  relatively  permanent,  and  have  the  oppormnity  to  train 
communication  and  coordination.  Communication  and  coordination  are  tasks  that  are  specific 
for  teams  (an  not  for  single  agents).  Many  authors  stress  the  importance  of  these  two  tasks 
(e.g..  Stout  &  Salas,  1993;  Rouse  et  al.,  1992;  Adams,  1995).  Communication  and 
coordination  are  discussed  in  more  detail  next. 

Communication.  Information  needed  to  perform  a  task  is  distributed  among  team  members  by 
means  of  communication.  Communication  is  not  always  a  question  and  answer  sequence. 
Typical  for  well-performing  teams  is  the  anticipation  of  information  needs,  so  as  to  present 
information  before  it  is  asked  for  (Rouse  et  al,  1992).  Serfaty  et  al.  (1993)  suggest  to  use  this 
feature  to  measure  team  performance  and  propose  an  anticipation  ratio:  the  relation  between 
provided  info  and  asked  info. 

According  to  Orasanu  (1993),  communication  is  very  task-directed  and  involves  plans, 
strategies,  intentions,  possibilities,  explanations,  warnings,  and  predictions.  Information  about 
task-related  needs,  positions,  roles,  needs,  responsibilities,  and  expectations  can  be  plamied  in 
advance.  For  successful  teams  this  takes  place  during  low  work  pressure  (Orasanu  &  Salas, 
1992;  Stout  &  Salas,  1993). 

Communication  can  be  categorized  in  direct  and  mediated  communication.  Direct 
communication  can  have  different  modalities  (speech,  touch  and  vision).  Chapanis  et  al. 
(1977)  investigated  the  effectiveness  of  direct  speech  and  vision  on  team  problem  solving 
(i.e.,  two  person  teams).  Teams  that  were  able  to  speak  with  each  other  solved  their  problems 
twice  as  fast  as  teams  that  communicated  without  speech  but  by  teletype  writing.  The  results 
should  be  handled  with  caution,  however,  since  the  tasks  (a  design  task  and  a  search  task) 
may  not  be  representative  for  all  tasks.  In  addition,  the  teams  consisted  of  two  members  only. 
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who  were  not  used  to  each  other.  Moreover,  the  team  tasks  were  not  carried  out  under  time 
pressure. 

Fussell  and  Benimoff  (1995)  distinguish  three  direct  communication  channels;  linguistic 
(verbal),  paralinguistic  (intonations  and  pauses),  and  non-linguistic  (non-verbal,  such  as 
gestures,  postures)  communication.  They  regard  communication  as  a  coordinated  activity  in 
which  all  three  channels  can  take  part.  Some  coordination  activities  are  turn  taking  (such  as 
by  a  falling  or  rising  intonation  characteristic  of  the  end  of  a  sentence),  and  back-channel 
responses  (such  as  “uh-huh”). 

Much  information  is  not  communicated  directly  but  is  mediated.  This  information  can  be 
textual  or  pictorial.  Team  members  can  type  in  a  message  and  send  it  to  each  other.  For  this 
way  of  communication,  the  message  should  be  clear  enough  to  be  understood,  without  any 
further  explanation.  Communication  with  text  and  pictures  can  also  be  used  in  a  more 
interactive  way,  within  a  shared  work  space.  One  can  make  conceptual  graphs  of  a  problem, 
use  conference  screens  or  general  plots  to  aid  communication,  etc.  Whittaker,  Geelhoed  and 
Robinson  (1993)  and  also  Tang  (1991)  found  that  such  shared  work  spaces  are  especially 
useful  when  the  task  has  a  substantial  graphical  component.  The  deictic  element  (i.e.,  pointing 
to  something  instead  of  describing  it)  is  generally  an  important  element  of  communication  (see 
e.g.  Cremers,  1996)  and  therefore  may  need  to  be  supported  in  mediated  communication. 

We  distinguish  four  different  categories  of  communication; 

•  Data  communication.  This  type  of  communication  is  highly  task  related  and  involves 
information  needed  to  complete  the  task.  People  may  explicitly  ask  for  data  information, 
or  give  this  information  voluntary. 

•  Task  communication.  During  a  task,  members  of  a  team  may  inform  each  other  about  the 
process  of  task  completion.  Team  members  log  their  activities  by  telling  each  other  what 
they  are  doing  on  a  particular  moment.  Also  team  members  correct  each  others  utterances 
and  express  their  workload  in  demanding  simations.  Furthermore,  evaluation  of  the  task 
performance  takes  place  and  team  members  coordinate  their  activities  by  agreeing  upon  an 
new  division  of  tasks.  Sometimes  a  team  member  takes  over  the  other  team  members’  task 
without  deliberation. 

•  Domain  knowledge  communication.  Team  members  speak  about  learned  rules  and 
strategies.  Also  team  members  evaluate  former  actions  and  formulate  new  strategies  to 
improve  their  performance.  This  type  of  communication  is  typical  for  intra-team  feedback. 

•  Meta-communication.  Meta-communication  is  communication  about  the  management  of 
communication  (Clarke,  1996),  or  in  other  words,  communication  for  coordinating 
communication.  An  example  is  asking  to  restrict  communication  to  the  necessary 
ingredients,  to  necessary  communication  lines,  or  to  status  changes  only. 

In  addition  to  the  above  described  categories  communication  control  take  place  as  well.  This 
communication  consists  of  utterances  which  indicated  if  a  message  was  received  and 
understand.  Examples  of  these  utterances  are  “okay”  or  “roger”. 
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Communication  can  be  supported  technically  with  electronic  conference  lines  and  point-to- 
point  connections. 

Coordination.  Our  definition  of  coordination  is  “a  task  in  which  other  tasks,  including 
communication,  are  organized  in  terms  of  work  allocation  over  time,  space,  and  agents”. 

Adams  (1995)  provides  a  framework  for  closed-loop  real-time  team  work  processes  in  which 
coordination  has  a  key  role.  He  distinguishes  external  and  internal  coordination.  External 
coordination  handles  inputs  of  external  agents  (supervisory  agents  or  cooperative  agents). 
Internal  coordination  deals  with  controlling  or  guiding  real-time  tasks. 

Cooperation.  The  term  cooperation  is  also  associated  with  teams.  Clarke  (1996)  uses  the 
following  definition  of  cooperation:  “acting  together  in  a  coordinated  way  [...]  in  the  pursuit 
of  shared  goals  [...]”  (p.59).  Clarke  (1996)  distinguishes  three  different  levels  of  outcome:  (i) 
efficiency,  where  cooperation  minimizes  the  effort  required  to  achieve  the  goal  (this  refers  to 
the  coordination  aspect,  such  as  synchronization  when  moving  a  piano);  (ii)  possibility,  where 
the  partners  can  achieve  goals  not  possible  for  one  person  to  achieve  (e.g.,  when  moving  a 
piano);  and  (Hi)  synergy,  where  the  cooperating  partners  can  together  achieve  a  different 
order  of  result  from  that  achievable  separately  (such  as  when  playing  quatre-mains). 

The  structure  of  teams.  Team  members  have  an  interrelationship.  They  differ  in  expertise, 
roles,  and  responsibilities.  The  performance  of  a  team  depends  on  how  the  team  is  structured. 
Hierarchically  organized  teams  perform  worse  than  other  teams  under  heavy  workload;  under 
low  and  medium  workload,  teams  with  overlapping  tasks  perform  better  (Urban,  Bowers, 
Monday  &  Morgan,  1995). 

Team  training.  Within  a  team,  learning  is  needed  to  get  used  to  each  others  way  and  to 
improve  coordination  and  communication  tasks.  One  way  is  to  let  the  members  of  a  team 
perform  the  tasks  of  their  colleagues.  Cross  training  does  not  increase  team  performance 
perse.  It  does  increase  the  knowledge  of  each  others’  information  needs  (Schraagen  &  Rasker, 
1996;  see  also  Schaafstal  &  Bots,  1997).  This,  however,  pays  only  in  hectic  situations  where 
seconds  can  make  a  whole  difference. 


2.6  Shared  mental  model  set 

Considering  the  overview  of  literature  presented  above,  the  concept  of  Shared  Mental  Model 

can  be  worked  out  in  more  detail.  We  will  propose  the  following  six  submodels: 

•  Shared  Situation  Model.  This  encompasses  a  description  of  the  current  situation,  possibly 
extended  with  history  information; 

•  Shared  Organizational  Resources  Model.  Here  an  overview  of  the  organizational  resources 
is  modelled,  such  as  manpower  (the  agents  with  known  knowledge  and  capacities), 
machines,  tools,  the  availability  and  deployment  of  assets,  etc.; 
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Fig.  1  The  shared  mental  model  set  from  the  perspective  of  the  organization. 


•  Shared  Goal  Model.  The  Goal  Model  states  a  description  of  the  situation  the  organization 
is  aiming  for  at  a  certain  point  in  time;  (e.g.,  safe  and  timely  arrival  at  point  p  and  time  t.) 

•  Shared  Task  Model,  expressing  the  tasks  to  carry  out  to  realize  the  goal  in  time; 

•  Shared  Communication  Model,  which  is  a  submodel  of  the  Task  Model;  in  this  model,  the 
conventions  of  communication  are  defined; 

•  Shared  Coordination  Model,  also  a  submodel  of  the  Task  Model; 
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•  Shared  Task  Allocation  Model,  describing  the  links  between  the  tasks  and  the 
organizational  resources;  important  for  team  functioning  is  to  know  each  other’s  position, 
function  and  expertise. 

Figure  1  presents  these  models  graphically  from  the  perspective  of  the  organization.  This 
figure  shows  an  organization  that  has  to  deal  with  the  situation  it  is  part  of.  The  situation  is 
described  in  the  Shared  Situation  Model.  The  current  situation  has  a  History.  Previous  actions 
of  the  organization  have  resulted  in  the  current  situation,  and  the  result  is  described  in  terms 
of  Feedback  and  Knowledge  of  Results.  The  organization  has  a  common  goal,  defined  in  the 
Shared  Goal  Model.  The  discrepancy  between  the  Goal  and  the  current  situation  defines  the 
job  the  organization  has  to  carry  out.  This  job  is  accomplished  by  performing  tasks,  described 
in  the  Shared  Task  Model. 


Fig.  2  The  Shared  Mental  Model  set  from  the  perspective  of  the  agent. 
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Two  sub-models  of  the  Shared  Task  Model  are  the  Shared  Communication  Model  and  the 
Shared  Coordination  Model.  The  organization  has  resources  available  to  realize  its  goal. 
These  are  defined  in  the  Shared  Resources  Model.  This  model  describes  the  agents  and  their 
possible  interdependence,  and  the  tools  and  machines  available.  When  agents  are 
interdependent,  they  form  a  team.  Based  on  previous  results  of  allocating  the  resources,  the 
current  resources  of  the  organization  are  allocated  to  the  tasks  defined  in  the  Shared  Task 
Model. 


From  the  perspective  of  an  agent,  the  model  is  presented  in  Figure  2.  An  individual  agent  has 
sensory,  motor  and  cognitive  capacities,  and  possesses  domain  knowledge  and  problem¬ 
solving  knowledge.  Both  knowledge  and  capacities  are  limited.  For  realizing  a  common  goal 
they  cannot  reach  by  themselves  agents  can  combine  knowledge  and  capacities  by  forming  a 
team.  For  working  together,  agents  need  to  communicate  and  to  coordinate  their  actions.  To 
enhance  communication  and  coordination,  they  need  shared  mental  models  of  their  situation, 
their  goal,  the  tasks  necessary  to  realize  the  goal,  the  resources  available  (including  the 
knowledge  and  capacities  of  their  fellow  agents),  and  how  the  resources  are  allocated  to  the 
tasks  needed  to  be  carried  out.  The  relation  between  the  agents  of  a  team  is  structured  (e.g.,  a 
hierarchical  relation  between  agents),  and  because  of  this  interrelation,  they  may  have 
responsibilities. 


2.7  Conclusions 

In  summary,  we  have  described  a  framework  of  team  knowledge  and  information  consisting 
of  various  elements:  the  situation,  the  organisation,  its  tasks,  and  the  allocation  of  tasks  to 
agents.  These  elements  guide  our  research  on  communication  and  coordination  further  in  the 
next  chapters. 

Based  on  the  models  proposed,  team  performance  may  be  enhanced  by  compensating  for  the 
limitations  of  the  individual  agents  as  follows: 

•  individual  agents  need  to  learn  individual  skills,  rules  and  knowledge  (both  problem 
solving  knowledge  and  domain  knowledge).  To  shorten  learning  time,  the  elements  of 
expertise  may  be  optimally  distributed  over  the  members  of  a  team; 

•  team  members  should  particularly  learn  how  to  communicate  and  coordinate; 

•  team  members  can  help  reduce  stress  by  noise  reduction,  anxiety  reduction,  and  by 
increasing  the  incentive; 

•  team  members  may  extend  the  working  memory  of  others; 

•  team  members  can  provide  the  skills,  rules  and  knowledge  (both  problem  solving 
knowledge  and  domain  knowledge)  in  their  long-term  memory  to  others; 

•  team  members  may  watch  each  other  on  tunnel  vision,  biases,  and  errors  (mistakes,  lapses 
and  mode  errors,  and  slips;  see  for  example  Wickens,  1992);  team  members  can  also 
distribute  problem  solving  subtasks;  this  is  proposed  by  the  work  of  Cohen  (1993).  Cohen 
states  that  not  all  problem  solving  tasks  are  based  on  recognition;  sometimes,  meta- 
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cognition  takes  place,  meaning  that  the  problem-solving  process  is  analysed  in  more  detail. 
This  process  can  be  done  by  more  than  one  person:  one  agent  can  take  on  the  role  of  the 
devil’s  advocate  and  can  generate  alternative  explanations  that  contradict  the  outcome  of 
pattern  recognition  performed  by  another  agent  (e.g.,  in  uncommon  situations,  when  the 
stakes  of  an  error  are  high,  when  some  time  is  left  over,  etc.). 

This  list  is  too  large  to  be  captured  in  one  study.  As  has  been  mentioned  in  §  1.3,  we  focus 
our  research  on  the  necessity  for  working  with  more  than  one  agent,  the  conditions  of 
communication  and  coordination,  and  the  distribution  of  knowledge  within  a  team. 


3  FIRE-FIGHTING  PARADIGM 

In  order  to  investigate  the  role  of  communication  and  coordination  in  team  decision  making 
we  will  simulate  situations  that  a  fire-fighting  organization  has  to  handle.  By  varying  the  way 
in  which  communication  can  take  place  within  this  organization,  insight  will  be  gained  in  how 
these  factors  contribute  to  team  performance.  First,  we  will  explain  the  situation.  Next,  we 
describe  the  organization.  Then,  we  analyse  the  tasks  to  handle  the  situation  in  detail.  After 
that,  we  describe  different  types  of  teams  that  should  carry  out  the  tasks. 


3.1  Situation 

The  situations  take  place  in  a  city.  This  city  consists  of  76  buildings  that  are  located  in  one  of 
four  sectors.  At  certain  points  in  time,  a  series  of  fires  threatens  the  city.  These  fires  take 
place  during  a  time  frame  of  12  time  periods.  Table  I  shows  these  time  periods. 

Table  I  A  scenario  of  12  time  periods  representing  the  situation  that  has  to  be  dealt  with. 


Period 

1  2  3  4  5  6 

7  8  9 

10  11  12 

Building  (76) 

house  school  house  house 

complex 

hospital 

Sector  (4) 

south  north  east  east 

east 

west 

Possible  casualties 

2  100  22 

10 

1000 

Units  needed 

13  11 

2 

5 

In  the  example  scenario,  first  a  house  gets  on  fire,  next  a  school,  again  a  house,  a  complex, 
and  finally  a  hospital.  When  a  building  burns  down,  a  number  of  lives  are  lost.  A  house  has 
two  potential  victims,  a  complex  10,  a  school  100,  a  factory  (not  shown  in  the  example 
scenario)  500,  and  finally,  a  hospital  1000.  To  save  the  lives,  fire-fighting  units  are  needed. 
For  a  house,  1  unit  suffices,  a  apartment  building  needs  2  units,  a  school  3,  a  factory  4,  and  a 
hospital  5. 
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3.2  Organization 

In  order  to  handle  this  situation,  a  fire-fighting  organization  has  been  set  up.  The  organization 
has  formulated  a  goal:  to  rescue  as  many  lives  as  possible.  The  resources  of  the  organization 
are  restricted:  only  6  fire-fighting  units  are  available.  In  a  dispatch  centre,  information  about 
the  fires  is  collected  and  the  fire-fighting  units  are  allocated. 

The  scenario  shows  that  the  most  important  building  to  save  is  the  hospital.  This  fire  can  be 
prevented  when  sufficient  units  are  located  at  the  hospital  at  the  onset  of  the  fire. 
Unfortunately,  the  fire-fighting  units  need  one  time  period  for  transportation  between  the 
fire-fighting  station  and  a  particular  building.  The  scenario  also  shows  that  not  enough  units 
are  available  after  having  assigned  the  units  to  the  fires  started  earlier.  Withdrawing  a  unit 
takes  one  time  period  as  well.  So,  when  the  hospital  (or  in  other  scenarios,  a  factory)  gets  on 
fire,  the  fire  fighters  will  always  be  too  late.  However,  the  organization  possesses  pattern 
knowledge  of  series  of  fires  that  can  be  used  to  predict  the  location  and  the  type  of  a  large 
building  that  will  get  on  fire.  When  three  small  buildings  in  one  sector  will  get  on  fire  (in  the 
example  scenario,  two  houses  and  a  appartment  building  in  sector  east),  a  targe  building  will 
get  on  fire  in  the  opposite  sector,  three  time  periods  later  (in  the  scenario,  a  hospital  in  sector 
west).  This  knowledge  can  be  used  to  (re)allocate  the  resources  in  time. 


3.3  Tasks 

We  analyse  the  fire-fighting  task  in  detail.  First,  we  decompose  fire  fighting  into  subtasks, 
and  allocate  the  subtasks  to  agents.  Next,  we  analyse  the  types  of  information  and  knowledge 
used  in  the  subtasks.  Based  on  this,  we  will  perform  a  time-line  analysis  for  fire  fighting  in  an 
experimental  environment  that  will  be  introduced  in  the  next  chapter.  The  task  analysis  and 
the  time-line  analysis  yields  a  normative  description  of  the  experimental  task.  This  will  be 
used  to  define  the  measurements  for  team  performance. 

3.3.1  Task  analysis 

Fire-fighting  can  be  divided  in  four  sub-tasks:  situation  awareness,  threat  assessment,  decision 
making  and  execution.  This  decomposition  is  based  on  the  generic  functions  suggested  in  van 
Delft  &  Schuffel  (1995).  These  sub-tasks  are  further  decomposed  into  a  task  hierarchy 
presented  in  Figure  3 .  The  type  of  information  and  knowledge  that  is  used  in  the  subtasks  are 
depicted  in  Figure  4.  We  will  explain  each  sub-task  in  detail. 
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Legend 

:  transfer 

task 

1  has  as  subtask 

transfer  task  of 

•  task  1 

agent 

agent 

Fig.  3  The  task-hierarchy  of  fire  fighting. 


Fig.  4  Information  and  knowledge  in  the  fire-fighting  task.  The  rounded  boxes  denote 
(sub-)tasks;  the  square  boxes  information;  the  arrows  data  dependencies;  and  the  double 
arrows  knowledge  dependencies. 


Situation  awareness  is  monitoring  the  outside  world  on  fires.  Situation  awareness  is 
decomposed  further  in  surveillance  (searching  for  potential  danger),  detection  (perceiving 
a  fire),  fire  identification  (identifying  the  location  and  the  type  of  fire,  that  is,  whether  it  is  a 
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house,  a  hospital,  etc.)  2L\\&fire  classification  (i.e.,  categorizing  the  fire-state:  whether  it  is  on 
fire,  burnt-down,  or  under  control).  Surveillance  uses  information  of  expected  features  (the 
location,  moment,  and  type  of  object  of  an  expected  fire)  to  guide  the  search  in  the  outside 
world.  Fire  detection  uses  information  in  the  outside  world  to  detect  an  object  on  fire.  Fire 
identification  describes  the  object  on  fire  (in  terms  of  location  and  type  of  object)  based  on 
knowledge  of  coordinates  and  object  types.  Finally,  fire  classification  uses  knowledge  about 
possible  fire  states  to  classify  the  severity  of  the  fire. 

The  second  task,  threat  assessment,  is  concerned  with  the  assessment  of  the  overall  situation, 
that  is,  the  relation  between  the  fires  in  terms  of  time  and  place.  This  is  used  to  predict  and 
anticipate  future  fires,  based  on  recognizing  patterns  of  the  buildings  on  fire,  which  is 
important  for  decision  making.  The  task  is  decomposed  in  predict  expected  time  of  fire, 
predict  object  type  and  predict  location.  The  task  uses  information  of  the  overall  situation 
(i.e.,  the  fires,  the  order  of  fires,  and  the  location  of  fires)  to  generate  features  of  an  expected 
fire  (the  expected  location  and  the  expected  type  of  object).  These  features  are  used  in  the 
situation  awareness  task  for  guiding  surveillance.  Situation  assessment  also  generates  a 
description  of  a  threat  (i.e.,  a  fire),  in  terms  of  expected  time,  location,  and  object.  This 
threat  description  is  used  in  the  decision  making  task  to  anticipate  deployment.  The  threat 
assessment  task  uses  knowledge  about  patterns  of  fires  for  predicting  new  fires. 

The  third  task,  decision  making,  results  in  a  decision  about  the  deployment  of  units.  The  task 
consists  of  determine  allocation  object,  determine  allocation  amount,  and  determine  allocation 
moment.  The  “determine  allocation  object”  task  determines  to  which  object  a  unit  should  be 
allocated,  or  from  which  object  a  unit  is  withdrawn.  This  task  uses  information  about  the 
situation  of  the  objects,  and  knowledge  about  priorities  (i.e.,  the  importance  of  objects  in 
terms  of  number  of  victims).  The  “determine  allocation  amount  task  is  concerned  with  how 
many  units  to  allocate  (or  withdraw).  This  task  is  carried  out  with  information  about  the 
current  amount  of  units  allocated  to  the  objects,  and  the  current  state  of  the  objects.  The  task 
also  uses  knowledge  about  the  deployment  duration  of  units.  The  “determine  allocation 
moment”  task  determines  at  what  moment  a  unit  should  be  allocated  or  withdrawn.  The  task 
uses  information  about  the  possible  threat  (especially  the  expected  time),  and  knowledge  of 
deployment  duration. 

The  fourth  and  final  task,  execution,  is  divided  in  transport  and  extinction.  Execution  is 
expected  to  have  impact  on  the  outside  world.  That  is  to  say,  there  is  no  deterministic  relation 
between  the  execution  of  a  cognitive  task  and  the  effect  of  it  on  the  outside  world,  only  a 
probabilistic  one  (the  intended  effect  may  be  reached).  The  impact  is  mediated  by  a  motor 
agent  which  receives  its  orders  from  the  execution  task.  The  execution  task  generates  the 
orders  using  the  decision  taken,  the  situation  model  and  the  fire  data. 
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Learning  fire-fighting 

As  with  all  tasks,  fire  fighting  should  be  evaluated  in  order  to  optimize  the  task.  Here  we 
present  a  tentative  model  of  how  this  evaluation  takes  place.  The  evaluation  is  based  on 
feedback  of  how  the  task  is  performed  and  on  quality  norms.  Task  optimization  will  take 
place  when  performance  and  quality  norms  are  not  in  agreement.  The  optimization  is  carried 
out  by  adapting  the  knowledge  about  the  domain  (such  as  about  existing  objects,  priorities 
etc.),  knowledge  about  how  to  carry  out  the  tasks  (i.e.,  the  order,  start  and  finish  of  tasks), 
and  knowledge  about  how  to  communicate.  The  knowledge  does  not  change  continuously 
during  task  execution,  only  after  having  received  feedback.  Changing  the  knowledge  also 
depends  on  the  available  time  to  reflect  on  the  performance. 

Background  knowledge  is  also  involved  in  task  execution.  Missing  knowledge  about  the 
domain,  the  task,  and  communication  is  compensated  by  prior  knowledge  that  one  has  about 
the  task,  or  a  similar  task.  Figure  5  shows  the  tasks  and  the  information  dependencies 
graphically. 


Fig.  5  Learning  fire-fighting. 


3.3.2  Task  allocation 

In  the  experimental  environment,  the  tasks  are  allocated  to  different  types  of  teams  consisting 
of  different  types  of  agents:  fire  guards,  fire-fighting  units,  the  observer  and  the  allocator. 
The  fire  guards  can  provide  information  about  how  many  units  are  needed:  the  fire 
classification.  This  information,  however,  should  explicitly  be  asked  for  by  the  observer.  The 
observer  takes  account  of  surveillance,  fire  detection,  fire  identification,  and  is  involved, 
together  with  the  fire  guard,  in  transferring  fire  classification  information  to  the  allocator.  The 
observer  is  involved,  together  with  the  allocator,  in  transferring  the  description  of  the 
situation.  The  allocator  carries  out  the  threat  assessment,  predicts  object  type,  predicts 
expected  time  of  fire,  and  predicts  location  tasks.  The  allocator  transfers  the  decision  to  the 
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units.  The  units  carry  out  transport  and  extinguish  the  fires.  The  allocation  of  the  tasks  and 
the  transfer  tasks  are  shown  in  Figure  4  as  well.  In  the  experimental  environment,  the  fire 
guard  and  the  fire  units  are  software  agents. 

In  the  next  section,  the  task  analysis  described  above  is  extended  with  a  normative  description 
of  the  task  by  means  of  a  time-line  analysis. 

3.3.3  Time-line  analysis 

By  performing  time-line  analysis,  we  will  establish  action  charts  for  the  fire-fighting  tasks.  A 
time-line  analysis  of  the  tasks  depends  on  how  the  tasks  and  the  knowledge  are  distributed 
among  the  agents  and  in  space.  Figures  6-9  present  the  action  charts  for  four  different 
conditions.  In  the  first  condition,  fire  fighting  is  carried  out  by  a  single  agent.  In  the  second 
condition,  a  fire-fighting  team,  consisting  of  an  observer  and  an  allocator,  perform  fire 
fighting  in  a  face-to-face  situation.  In  the  third  condition,  the  two  members  of  the  fire-fighting 
team  are  physically  separated,  and  the  team  does  not  share  knowledge  of  how  to  assess  the 
threat;  only  the  allocator  possesses  this  knowledge.  Communication  is  restricted  to  a 
minimum:  only  the  information  to  perform  the  tasks,  such  as  the  buildings  that  are  on  fire  and 
in  threat,  are  exchanged.  In  the  fourth  condition,  the  situation  differs  at  one  point  with  the 
previous  condition;  both  the  observer  and  the  allocator  have  threat  assessment  knowledge 
available. 

The  action  charts  only  show  the  time  periods  7 ,  8  and  9 .  In  these  time  periods ,  task 
performance  is  most  critical  and  therefore  the  focus  of  the  present  investigation.  Because  the 
large  buildings  will  get  on  fire  at  a  fixed  time  (in  time  period  10),  the  action  charts  need  only 
to  describe  some  critical  time  periods  in  advance.  The  first  six  time  periods  are  not  critical 
(i.e.,  sufficient  time  is  available  to  carry  out  the  tasks).  At  the  beginning  of  time  period  7,  the 
last  piece  of  information  becomes  available  for  recognizing  a  pattern  that  is  used  for  searching 
and  finding  the  large  building  under  threat.  Assessing  the  threat  and  finding  the  threatened 
building  too  late  delays  (re)allocation  of  the  fire  units,  which  has  serious  consequences  for 
being  in  time  to  rescue  the  large  building. 

Condition  1  :  individuals 

In  the  condition  presented  in  Figure  6,  the  single  agent  starts,  at  the  beginning  of  time  period 
7,  with  a  situation  awareness  task  (denoted  by  SA).  He  or  she  detects  a  building  on  fire  and 
identifies  the  object  type.  Knowing  what  the  previous  buildings  were,  he  recognizes  a  pattern 
(marked  by  TA),  and  is  now  able  to  predict  the  object  type  and  the  location  of  the  fire  that  is 
expected  to  start  in  time  period  10.  Next,  he  starts  to  determine  how  many  fire  units  he  needs 
to  send  to  the  fire,  and  if  not  enough  are  available  in  the  fire  station,  from  which  objects  they 
need  to  be  withdrawn  (indicated  by  DM).  Also,  the  fire-units  have  to  be  notified  (referred  by 
C,  meaning  communication).  Now,  a  surveillance  task  can  start  in  which  he  will  search  for 
the  threatened  building  of  a  certain  type  and  at  a  certain  sector  (SA).  After  the  threatened 
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building  has  been  found,  the  building  is  added  to  a  plot  board  (C).  Finally,  he  allocates  the 
available  fire  units  to  the  threatened  building  (DM)  and  has  to  inform  the  fire  units  (C). 
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Fig.  6  Individual  condition. 


The  agent  has  to  work  with  two  important  deadlines.  Since  the  fire-fighting  units  only  receive 
messages  at  the  beginning  of  a  time  period,  and  the  remaining  time  period  is  needed  for 
transportation,  it  is  essential  that  decisions  about  withdrawing  units  (in  time  period  7)  and 
about  allocating  units  (in  time  period  8)  are  communicated  in  time,  that  is,  before  the  start  of 
a  new  period.  Otherwise,  transport  is  delayed  with  a  full  time  period.  The  most  critical  task  is 
the  surveillance  task  (the  second  SA  task  in  the  figure).  When  the  building  under  threat  is  not 
found  in  time,  the  fire-fighting  units  will  arrive  too  late  at  the  building,  causing  many 
casualties.  Therefore,  it  is  important  to  start  this  task  as  early  as  possible.  In  Figure  6,  the 
length  of  the  second  SA  task  represents  the  available  time  for  searching;  how  much  time  the 
task  takes  depends  on  the  chance  of  finding  the  threatened  building.  The  duration  of  the  other 
tasks  are  always  the  same. 

Condition  2:  unrestricted  communication 

Figure  6  clearly  shows  that  the  tasks  are  carried  out  sequentially.  One  way  to  start  earlier  with 
the  search  for  the  threatened  building  is  to  carry  out  tasks  in  parallel.  To  do  this,  a  second 
agent  is  needed.  Figure  7  shows  this  condition. 
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Fig.  7  Team  with  unrestricted  communication. 


The  observer  starts  with  situation  awareness.  The  last  piece  of  the  pattern  is  communicated  to 
the  allocator,  and  the  observer  can  continue  directly  with  threat  assessment,  and  with 
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searching  for  the  threatened  building,  once  the  building  type  and  location  is  determined.  In 
parallel,  the  allocator  has  assessed  the  threat  and  withdraws  fire-fighting  units.  Compared  to 
the  previous  condition,  there  is  some  overhead  for  necessary  communication,  but  the  second 
SA  task  can  start  earlier,  and  more  time  is  left  for  finding  the  threatened  building. 


Condition  3:  restricted  communication  without  shared  knowledge 


Figure  8  shows  a  condition  in  which  team  members  can  communicate  and  coordinate  in  a 
restricted  way.  Only  necessary  ingredients  are  communicated.  As  one  can  see,  an  important 
dependency  arises:  the  observer  has  to  wait  at  least  for  the  allocator  having  assessed  the  threat 
(the  allocator  may  also  follow  the  strategy  first  to  decide  to  withdraw  units,  and  having 
notified  this  to  the  fire  units  before  communicating  the  threatened  building  type  and  sector  to 
the  observer;  this  in  order  to  send  the  message  to  the  units  before  the  deadline  is  reached). 
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Fig.  8  Team  with  restricted  communication  without  shared  knowledge. 


Condition  4:  restricted  communication  with  shared  knowledge 

Figure  9  shows  a  condition  in  which  the  dependency  of  condition  3  is  minimized  by  providing 
the  observer  with  pattern  knowledge;  threat  assessment  can  now  be  carried  out  by  both  team 
members. 


90sec 


120sec 


120sec 


|SA- 

C 

TA 

SA 

C 

c 

TA 

DM 

DM 

; 

TRANSPORT  ||| 

C 

TRANSPORT  || 

Fig.  9  Team  with  restricted  communication  with  shared  knowledge. 
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3.4  Hypotheses 

We  have  now  all  the  ingredients  at  our  disposal  to  translate  our  three  research  questions  stated 
in  §  1.3  into  testable  hypotheses.  The  framework  presented  in  chapter  2  tells  us  that  when  an 
agent  is  not  able  to  perform  a  job  alone,  he  can  form  a  team;  the  task  analysis  in  the  previous 
section  predicts  that  for  the  fire-fighting  task,  an  individual  indeed  performs  less  well  than  a 
team.  Further,  our  task  analysis  does  not  predict  differences  between  unrestricted  and 
restricted  communication.  However,  the  framework  describes  that  teams  conununicate  not 
only  data,  but  also  domain  knowledge  and  task  information.  Also,  the  framework  indicates 
that  members  rely  on  knowledge  to  carry  out  their  tasks;  the  task  analysis  predicts  that  shared 
knowledge  decreases  interdependency  among  team  members.  This  all  leads  us  to  formulate 
the  following  three  hypotheses: 

1  Individuals  can  not  perform  the  task  well  because  they  are  not  able  to  perform  crucial  task 
elements  in  parallel. 

2  Communication  restricted  to  data  exchange  leads  to  a  decrease  in  team  performance,  since 
other  types  of  communication  contribute  to  team  performance  as  well. 

3  Restricting  shared  knowledge  to  one  team  member  leads  to  interdepedency  increase  and 
therefore  to  team  performance  decrease. 


4  METHOD 

This  section  gives  a  description  of  the  method  used.  In  total,  the  experiment  consisted  of  the 
four  conditions  described  in  §  3.3. 


4.1  Subjects 

77  students  of  the  University  of  Utrecht  participated  in  the  experiment.  Their  mean  age  was 
23  years  (o=3).  11  subjects  participated  in  the  individual  condition  (7  male  and  5  female)  and 
66  subjects  participated  in  the  teams  condition  (34  male  and  32  female).  Each  team  consisted 
of  2  subjects.  22  subjects  participated  in  the  unrestricted  communication  condition.  Each  team 
consisted  of  two  subjects  of  the  same  sex  (6  male  teams  and  5  female  teams).  22  subjects 
participated  in  the  restricted  communication  condition  without  shared  knowledge  and  22 
subjects  participated  in  the  restricted  communication  condition  with  shared  knowledge.  For 
both  conditions  with  restricted  communication,  male  and  female  were  divided  as  follows:  6 
teams  with  subjects  of  the  same  sex  (3  male  and  3  female)  and  5  mixed  teams  (2  teams  with  a 
male  allocator  and  3  teams  with  a  female  allocator).  The  subjects  were  paid  Dfl.  60,=  and 
were  informed  that  they  had  a  chance  of  receiving  a  bonus  of  Dfl.  40,  = .  This  bonus  was 
given  to  the  best-performing  individual  and  to  each  subject  of  the  best-performing  team  for 
each  condition. 
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4.2  Task 

Subjects  were  required  to  imagine  that  they  were  operators  of  a  dispatch  centre  of  a 
fire-fighting  organisation  of  a  city,  as  outlined  in  §  3.3.1.  The  mission  of  the  subjects  was  to 
fight  fires  in  order  to  keep  the  number  of  casualties  as  low  as  possible.  A  computer 
simulation,  especially  developed  for  this  type  of  experimental  research  (see  Schraagen,  1995), 
represented  the  functions  of  the  dispatch  centre.  The  computer  simulation  was  designed  to 
support  the  tasks  of  an  observer  and  an  allocator.  A  separate  interface  was  available  for  each 
of  them.  For  the  detection  task  the  subject  was  presented  with  a  map  of  the  imaginary  city  in 
which  the  buildings  were  shown  as  icons.  The  subjects  were  informed  about  the  state  of  the 
buildings  by  graphical  means  (red  for  fire,  black  for  burn  down,  and  green  for  extinguished). 
The  four  sectors  of  the  city  were  represented  by  boundaries  on  the  map.  Every  building  in  the 
city  had  its  own  identifier  annotated,  for  example  house_A(I)  (the  roman  character  informed 
the  subject  about  the  sector  where  the  building  was  located).  When  the  status  of  a  building 
changed,  the  observer  could  inform  the  allocator  about  this  by  first  clicking  on  the  icon.  This 
provided  him  or  her  with  information  about  the  fire  (numbers  of  units  needed,  possible 
danger).  Next,  to  send  this  information  to  the  allocator,  a  standardised  e-mail  function  was 
available. 

The  allocator  could  assign  the  six  fire  units  by  first  loading  the  message  of  the  observer  into  a 
plot  board,  by  clicking  on  a  button.  Next,  he  or  she  had  to  indicate  to  which  building  on  the 
plot  board  a  fire-fighting  unit  has  to  be  send  (or  withdrawn).  The  number  of  fire  units  which 
had  to  be  allocated,  depending  on  the  type  of  building  and  the  progress  of  extinction,  was 
represented  on  the  plot  board  as  well.  The  information  about  the  progress  of  extinction,  and 
subsequently  the  number  of  fire  units  needed,  were  sent  by  the  observer  by  standardized 
e-mail.  In  turn,  the  allocator  could  inform  the  observer  by  standardized  e-mail  about  the 
number  of  fire-units  which  were  allocated  to  a  building.  The  standardised  e-mail  could  also  be 
used  for  asking  questions  about  the  number  of  fire-units  needed  or  the  number  of  fire-units 
allocated.  Communication  which  took  place  via  the  e-mail  was  purely  task  related.  Therefore, 
the  standardised  e-mail  should  be  regarded  as  data  communication  only. 

Recall  from  chapter  3  that  to  keep  the  number  of  casualties  as  low  as  possible,  fires  should  be 
detected  as  quickly  as  possible  and  sufficient  fire  units  should  be  allocated  in  time.  Recall  also 
that  beside  some  small  buildings  like  houses  or  a  school,  each  scenario  contained  at  least  one 
burning  factory  or  one  burning  hospital.  Minimising  the  number  of  casualties  was  highly 
dependent  on  how  well  subjects  succeeded  in  extinguishing  the  fires  of  these  large  buildings. 
Recognising  a  certain  pattern  of  fires  in  small  buildings  made  it  possible  to  predict  a  fire  in  a 
large  building.  Anticipation  of  these  fires  (on  basis  of  the  pattern)  was  necessary  for  achieving 
a  low  number  of  casualties. 

A  pattern  was  made  up  of  three  small  buildings  (houses  and  apartments)  which,  one  at  a  time, 
had  been  on  fire  in  the  same  sector.  The  type  of  buildings  and  the  sector  where  the  pattern 
was  presented,  predicted  a  possible  future  fire  in  a  large  building  elsewhere.  The  principle 
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was  that  a  pattern  of  three  small  buildings  (houses  or  apartment  buildings)  in  one  sector 
predicted  a  fire  in  a  large  building  located  in  the  diagonally  opposite  sector.  In  order  to  find 
this  threatened  building,  subjects  had  to  search  for  a  special  warning.  Searching  was  done  by 
scanning  the  map,  by  means  of  clicking  each  building  with  the  computer-mouse.  When  the 
warning  was  found  the  subjects  had  information  about  the  exact  period  that  the  threatened 
building  would  be  on  fire.  To  save  this  building,  fire  units  had  to  be  reallocated  from  small 
buildings  toward  the  large  building.  The  exact  type  of  large  building  (factory  or  hospital) 
could  be  predicted  by  the  final  building  of  the  pattern  (a  house  meant  that  a  factory  would  go 
on  fire,  an  apartment-building  meant  that  a  hospital  would  go  on  fire). 

In  some  scenarios  the  e-mail  was  distorted  (only  during  the  experimental  task,  not  during 
training).  This  was  done  by  adding  a  different  label  to  a  building  (for  example, 
complex_F(IV)  instead  of  complex_S(I)).  The  distortion  of  the  e-mail  was  fixed  as  follows. 
First,  only  the  mail  transferred  from  the  detection  screen  to  the  allocation  screen  was 
distorted.  Second,  only  the  mail  concerning  a  building  in  the  middle  of  a  pattern  of  three 
small  buildings  was  distorted.  Third,  the  distortion  only  influenced  information  concerning  the 
identification  and  the  sector,  thus  information  concerning  the  type  of  building  remained  the 
same.  Consequently,  it  was  no  longer  possible  to  predict  an  upcoming  fire  on  the  basis  of  the 
information  received  by  the  e-mail  in  the  allocation  task. 

Besides  some  scenarios  with  distorted  e-mail,  there  were  also  “non-routine”  scenarios 
developed  (only  during  the  experimental  task,  not  during  training).  In  these  scenarios  the 
newly  learned  knowledge  about  patterns  was  not  applicable.  This  was  due  to  the  fact  that  the 
location  of  the  fire  was  different  than  would  be  expected  on  the  basis  of  the  pattern  (instead  of 
occurring  in  the  diagonally  opposite  sector,  the  fire  would  occur  in  one  of  the  three  remaining 
sectors).  Nevertheless,  the  prediction  with  regard  to  the  type  of  building  (factory  or  a 
hospital)  remained  intact. 

An  elaborate  description  of  the  experimental  task  can  be  found  in  Schraagen  and  Koster 
(1996). 


4.3  Procedure 

The  experiment  was  divided  into  four  parts:  three  training  sessions  and  the  actual  experiment. 
The  first  training  session  was  intended  to  teach  subjects  to  use  the  interface  of  the  simulation 
of  the  fire-fighting  station.  Therefore  subjects  had  to  read  a  written  instruction.  This 
instruction  contained  global  information  about  the  mission  of  the  task  and  specific  information 
how  to  use  the  computer-mouse  for  clicking  different  buttons  on  the  screen.  Also  this  part  of 
the  written  instruction  contained  some  exercises.  These  exercises  had  to  be  performed  with  a 
stationary  screen  (at  that  instance  a  scenario  was  not  running).  In  that  way  all  aspects  of  the 
screen  and  the  buttons  used  were  explained  one  step  at  the  time.  Compared  with  the  subjects 
in  the  three  teams  conditions,  the  training  was  different  for  the  subjects  in  the  individual 
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condition.  Subjects  in  the  individual  condition  had  to  control  both  screens  simultaneously. 
That  is,  they  had  to  perform  the  detection  task  as  well  as  the  allocation  task.  To  control  both 
screens  subjects  had  at  their  disposal  a  specially  designed  computer-mouse  whose  cursor 
could  be  moved  from  one  screen  to  the  other.  In  contrast  with  the  individuals,  each  subject  in 
the  three  team  conditions  was  assigned  to  either  the  detection  task  (observer)  or  the  allocation 
task  (allocator). 


The  second  training-session  was  intended  to  teach  the  subjects  how  to  perform  the  task. 
During  the  second  training-session  the  subjects  were  presented  with  16  scenarios.  They 
started  with  moderately  difficult  scenarios  and  ended  with  difficult  scenarios.  A  scenario  was 
more  difficult  as  the  number  of  buildings  set  on  fire  increased.  Furthermore  the  succession  of 
fires  took  place  in  a  shorter  time  notice.  As  a  result  the  subjects  had  to  make  decisions  under 
high  time  pressure  during  the  difficult  scenarios.  Also  the  subjects  had  to  prioritise  the 
buildings  as  the  number  of  fire-units  was  limited.  The  subjects  in  the  three  teams  conditions 
were  trained  separately  and  were  not  allowed  to  talk  to  each  other.  Their  teammates  were 
played  by  an  automated  software  agent.  These  agents  (one  for  the  detection  task  and  one  for 
the  allocation  task)  were  modelled  in  such  a  manner  that  their  behaviour  was  similar  to  a 
human  performer.  In  this  way  it  was  possible  to  train  each  subject  under  controlled 
conditions,  since  subjects  did  not  work  together  and  they  were  not  susceptible  to  each  others’ 
behaviour  and  possible  mistakes.  Moreover,  each  software  agent  always  behaved  in  the  same 
matter. 


After  finishing  the  second  training-session,  subjects  were  presented  with  another  written 
instruction.  This  instruction  was  intended  to  teach  the  subjects  how  to  recognise  patterns.  The 
information  concerning  pattern  recognition  was  different  for  the  restricted  communication 
condition  without  shared  knowledge  in  comparison  with  the  teams  of  the  other  conditions.  In 
the  condition  without  shared  knowledge  only  the  allocator  was  informed  about  patterns,  while 
in  the  other  conditions  both  team  members  were  informed  so  that  the  observer  had  the  same 
task  knowledge  as  the  allocator.  Subjects  trained  in  pattern  recognition  had  a  list  with  patterns 
and  threatened  buildings  at  their  disposal,  which  they  could  use  during  the  experimental  task. 
After  reading  the  instruction  the  subjects  were  trained  in  another  16  scenarios.  These 
scenarios  were  developed  in  order  to  train  the  subjects  in  applying  the  newly  learned 
knowledge  about  patterns.  The  subjects  in  the  teams  condition  were  trained  with  the  aid  of  the 
automated  software  agents  which  were  also  used  during  the  second  training-session.  The 
experimental  task  was  preceded  by  a  brief  written  instruction.  This  instruction  informed 
subjects  about  the  differences  between  the  training-sessions  and  the  experimental  task.  The 
subjects  were  told  that  the  e-mail  that  transferred  from  the  detection  screen  to  the  allocation 
screen  could  in  some  cases  be  distorted.  Subjects  were  also  told  that  their  knowledge  with 
regard  to  the  location  of  the  threatened  building  was  not  always  usable.  Though  a  pattern 
always  rightfully  warned  the  subject  about  the  type  of  building  (whether  it  was  a  factory  or  an 
hospital)  which  could  be  on  fire,  the  location  could  be  different  than  would  have  been 
expected  on  the  basis  of  the  pattern. 
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The  experimental  task  contained  16  scenarios,  made  up  of  12  time  periods  of  15  seconds 
each.  Each  team  or  individual  was  presented  with  identical  scenarios  in  a  fixed  order.  The 
first  four  scenarios  consisted  of  so  called  “routine  scenarios”.  In  these  scenarios  the  e-mail 
was  not  distorted  and  the  knowledge  of  the  subjects  concerning  the  patterns  was  always 
usable.  The  next  four  scenarios  were  also  routine  scenarios.  In  these  scenarios,  however,  the 
e-mail  was  distorted.  The  following  four  scenarios  contained  so  called  “non-routine”  with 
undistorted  e-mail.  The  experimental  task  ended  with  four  non-routine  scenarios  with 
distorted  e-mail.  The  reason  that  scenarios  were  presented  in  a  fixed  order  was  that  in  this 
way  subjects  immediately  could  apply  their  newly  learned  knowledge  about  patterns  without 
any  limitations.  If  subjects  were  directly  confronted  with  non-routine  scenarios  or  scenarios 
with  distorted  e-mail,  it  might  have  discouraged  subjects  to  use  this  knowledge.  After  all,  in 
these  scenarios  the  knowledge  about  patterns  did  not  contribute  to  their  performance.  Because 
of  this,  there  was  a  possibility  that  subjects  in  a  later  stage  of  the  experimental  task  no  longer 
used  the  knowledge  about  patterns,  even  if  they  eventually  where  presented  with  scenarios 
were  this  knowledge  was  applicable. 


4.4  Design 

Within  subjects  conditions 

The  distortion  of  the  standardised  e-mail  was  manipulated  within  subjects.  All  subjects  were 
presented  with  8  scenarios  without  distorted  e-mail  and  8  scenarios  with  distorted  e-mail. 
Likewise,  routine  and  non-routine  scenarios  were  manipulated  within  subjects.  All  subjects 
were  presented  with  8  routine  scenarios  and  8  non-routine  scenarios.  In  this  way  four 
“blocks”  of  scenarios  were  formed:  two  blocks  of  routine  scenarios  with  and  without  distorted 
e-mail  and  two  blocks  of  non-routine  scenarios  with  and  without  distorted  e-mail. 

Between  subjects  conditions 

1  Individuals.  Individuals  had  to  carry  out  both  the  detection  as  well  as  the  allocation  task. 
With  a  specially  designed  computer  mouse  they  could  control  the  feamres  on  the  detection 
screen  as  well  as  the  allocation  screen. 

2  Unrestricted  communication.  In  these  teams  both  the  allocator  and  the  observer  were 
informed  about  patterns.  Subjects  were  placed  in  the  same  room  and  communication  took 
not  only  place  via  the  standardised  e-mail  facilities,  but  also  face-to-face. 

3  Restricted  communication  without  shared  knowledge.  In  these  teams  only  the  allocator  was 
informed  about  patterns.  Subjects  were  physically  separated  and  communication  took  place 
strictly  via  the  standardised  e-mail  facilities. 

4  Restricted  communication  with  shared  knowledge.  In  these  teams  both  the  allocator  and  the 
observer  were  informed  about  patterns.  Subjects  were  physically  separated  and 
communication  took  place  strictly  via  the  standardised  e-mail  facilities. 
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4.5  Dependent  variables 

Manipulation  check 

In  order  to  check  whether  subjects  performed  according  to  the  instructions,  a  manipulation 
check  was  carried  out  on  whether  the  observers  searched  for  the  correct  type  of  building  in 
the  correct  sector,  after  having  interpreted  the  last  building  of  the  pattern. 

Performance  measures 

The  analysis  of  the  fire-fighting  paradigm  described  provided  us  with  information  concerning 
the  critical  time  frames  of  the  fire-fighting  task.  This  information  was  used  to  establish 
objective  performance  measures  of  team  functioning.  The  main  performance  measures  were, 
first,  whether  the  allocator  had  pulled  back  sufficient  resources  in  time  frame  8,  and,  second, 
whether  in  time  frame  10  sufficient  resources  were  allocated  to  either  the  hospital  or  the 
factory. 

Timing  of  actions  and  communication  of  messages 

In  order  to  gain  insight  when  subjects  performed  an  action  or  sent  each  other  messages  with 
the  e-mail  the  timing  of  actions  and  communication  of  messages  was  recorded.  This  was  done 
for  the  critical  time  frames  7,  8,  9  and  10.  Starting  with  the  final  building  of  the  pattern,  the 
processes  of  sending  and  reading  messages  and  the  start  of  actions  were  measured.  The 
following  variables  were  recorded: 

1  Time  at  which  observer  sends  last  building  of  pattern  to  allocator. 

2  Time  at  which  allocator  reads  message  about  last  building  of  pattern. 

3  Time  at  which  observer  starts  searching  for  threat  message  of  factory  or  hospital. 

4  Time  at  which  observer  sends  threat  message  to  allocator. 

5  Time  at  which  allocator  reads  threat  message. 

Observer-rating 

During  the  experimental  task  a  knowledgeable  observer  rated  the  verbal  communication 
which  took  place  between  the  subjects  in  the  face-to-face  condition.  A  specially  designed  form 
was  used  to  rate  the  communication  into  different  categories.  The  following  categories  were 
used: 

•  Data  communication.  Information  about  fires,  the  location,  type  of  building,  and  the 
number  of  fire-units  was  rated  as  “data  communication”.  There  were  distinct  ratings  for 
questions  and  answers. 

•  Task  communication.  This  was  rated  into  6  subcategories.  First,  communication  about  the 
present  actions  members  of  a  team  were  carrying  out  was  rated  as  “logging”.  Second, 
when  members  of  a  team  were  correcting  each  others  utterances,  this  was  scored  as 
“correction”.  Third,  team  members  indicated  their  workload,  this  was  scored  as 
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“workload  expression”.  Fourth,  when  team-members  mentioned  how  well  they  performed, 
this  was  rated  as  “evaluation”.  Fifth,  communication  in  order  to  agree  upon  a  new  division 
of  tasks  was  rated  as  “task  reallocation”.  Sixth,  when  one  team-member  told  the  other 
team-member  without  deliberation,  which  action  he  or  she  should  carry  out,  this  was  rated 
as  “task  adaption” . 

•  Domain  knowledge  communication.  This  category  was  used  when  members  of  a  team 
spoke  with  each  other  about  learned  rules  and  strategies  in  order  to  improve  their 
performance. 

•  Meta-communication.  Communication  involving  conununication  was  rated  as  “meta¬ 
communication”  . 

•  Remaining  communication.  Communication  which  was  unclear  of  could  not  be  rated  in  the 
existing  categories  (for  example  social  talk)  was  rated  as  “remaining  communication”. 

In  addition  to  the  above  described  categories  communication  control  was  also  rated.  This  was 

rated  for  data  communication,  task  communication  and  domain  knowledge  communication 

and  consisted  of  utterances  which  indicated  if  a  message  was  received  and  understood. 

Examples  of  these  utterances  are  “okay”  or  “roger”. 


5  RESULTS 

5.1  Relevant  comparisons 

In  order  to  test  the  main  hypotheses,  three  pairwise  comparisons  between  the  four  groups 

were  carried  out: 

1  In  order  to  test  whether  the  fire-fighting  task  was  a  true  team  task,  the  unrestricted 
communication  condition  was  compared  with  the  individual  condition.  The  rationale 
behind  this  comparison  was  that  both  individuals  as  well  as  the  team  members  of  the 
unrestricted  communication  condition  shared  pattern  knowledge  and  were  unrestricted  in 
their  information  exchange  (under  the  assumption  that  individuals  shared  knowledge  with 
themselves  and  communicated  with  themselves).  The  only  difference  between  the  two 
conditions,  then,  was  that  the  individuals  had  to  carry  out  certain  crucial  task  elements  in  a 
serial  fashion,  whereas  the  teams  could  carry  out  these  task  elements  in  parallel. 

2  In  order  to  test  the  effect  of  unrestricted  communication,  the  teams  of  the  unrestricted 
communication  condition  and  the  teams  of  the  restricted  communication  condition  with 
shared  knowledge  were  compared.  In  both  groups,  pattern  knowledge  was  shared  by  the 
observers,  the  only  difference  being  that  the  physically  separated  teams  were  restricted  in 
their  communication  to  task  level  information  exchange  (i.e.,  data  communication), 
whereas  the  free  communication  group  was  unrestricted  in  its  information  exchange. 
Nevertheless,  in  both  conditions  teams  had  the  possibility  for  an  optimal  performance. 

3  In  order  to  test  the  effect  of  shared  knowledge,  teams  of  the  restricted  communication 
conditions  with  shared  knowledge  and  without  shared  knowledge  were  compared.  In  both 
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conditions  communication  took  place  via  standardised  e-mail  and  subjects  were  physically 
separated,  the  only  difference  being  that  the  observers  without  shared  knowledge  did  not 
have  any  pattern  knowledge,  whereas  the  observers  with  shared  knowledge  did  have 
pattern  knowledge. 


5.2  Manipulation  check 

The  results  showed  that  all  subjects,  in  all  conditions,  searched  for  the  correct  type  of 
building,  indicating  that  instructions  were  well-understood.  In  most  cases  subjects  also 
searched  in  the  correct  sector  (i.e.,  the  one  opposite  to  where  the  last  building  of  the  pattern 
started  burning).  Averaged  across  the  16  scenarios,  the  percentages  of  teams  searching  in  the 
correct  sector  were  82%  for  the  unrestricted  communication  condition,  88%  for  the  restricted 
communication  condition  with  shared  knowledge,  and  89%  for  the  individuals.  The  remaining 
teams  probably  learned,  in  the  non-routine  conditions,  to  search  in  some  sector  other  than  the 
opposite  one.  This  adaptive  learning  effect  was  apparent  only  in  the  final  block  of  four 
scenarios  (non-routine  scenario  and  distorted  e-mail).  The  observers  in  the  restricted 
communication  group  without  shared  knowledge  were  dependent  on  the  allocator  for 
directions  on  where  to  search.  In  cases  of  e-mail  breakdowns,  the  allocator  did  not  perceive  a 
pattern  and,  hence,  could  not  direct  the  observer  to  a  sector.  Consistent  with  this 
manipulation,  in  this  group  only  64%  of  the  observers  searched  in  the  correct  sector,  this 
percentage  being  higher  in  the  non-distorted  e-mail  scenarios  (76%)  than  in  the  distorted  e- 
mail  scenarios  (51%). 

We  may  conclude  from  these  results  that  subjects  acted  according  to  instructions. 


5.3  Performance  measures 

Timeframe  8 

Two  main  strategies  could  be  discovered  for  assembling  sufficient  resources  in  time  frame  8; 
the  resources  could  either  be  pulled  back  from  other  buildings  or  they  could  be  kept  in 
reserve  all  the  time  (i.e.,  teams  chose  to  let  other  buildings  prior  in  the  scenario  burn  down). 
The  second  strategy  is  clearly  less  optimal  than  the  first,  since  buildings  unnecessarily  burn 
down.  However,  both  strategies  are  better  than  the  third  alternative:  not  having  sufficient 
resources  available.  For  each  scenario,  each  team  was  classified  as  performing  in  either  of 
three  categories:  sufficient  resources  after  pulling  back,  sufficient  resources  still  available, 
and  insufficient  resources. 

The  results  are  shown  in  Figure  10.  A  loglinear  model  was  fitted  to  the  data.  The  best-fitting 
model  only  contained  the  variables  “condition”,  “category  of  resource  allocation”,  and  the 
interaction  between  the  two.  The  variable  “block”  (i.e.,  the  within  subjects  manipulation),  nor 
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any  interaction  with  this  term,  was  not  significant.  Hence,  there  were  no  effects  of  distorted  e- 
mail  or  of  non-routine  scenarios. 


knowledge  knowledge 

Fig.  10  Performance  measurements  during  time  frame  8. 


There  was  a  significant  interaction  when  comparing  the  individuals  with  the  unrestricted 
communication  teams,  x^(20) =52.79,  p<  .001.  Inspection  of  Figure  10  shows  that  this  result 
is  caused  by  the  unrestricted  communication  group  having  assembled  sufficient  resources  in 
more  scenarios  than  the  individuals  (sufficient  resources  were  assembled  in  time  frame  8  by 
46%  of  the  unrestricted  communicating  teams  versus  26%  of  the  individuals).  The  interaction 
was  also  significant  when  comparing  the  unrestricted  and  the  restricted  communication  groups 
(both  with  shared  knowledge),  x^(20) =47.98,  ;i<.001.  Inspection  of  Figure  10  shows  that 
this  result  is  caused  by  the  group  with  unrestricted  communication  having  assembled  sufficient 
resources  in  more  scenarios  than  the  group  with  restricted  communication  (sufficient 
resources  were  assembled  in  time  frame  8  by  46%  of  the  unrestricted  communicating  teams 
versus  23%  of  the  restricted  communication  teams  with  shared  knowledge).  The  interaction  of 
condition  by  resource  allocation  was  not  significant  when  comparing  the  restricted 
communication  groups  with  and  without  shared  knowledge,  x2(20)=23.26,  p>  .10,  indicating 
that  in  both  groups  the  allocator  was  unable  to  assemble  sufficient  resources  in  time  frame  8. 

Time  frame  10 

In  time  frame  10,  the  large  building  (hospital  or  factory)  started  to  burn.  In  order  to  be  able  to 
effectively  fight  this  fire,  the  required  number  of  resources  needed  to  be  present  at  the  site  of 
the  large  building  at  the  start  of  time  frame  10.  Three  categories  could  be  distinguished: 
sufficient  resources  were  available  at  the  start  of  time  frame  10,  sufficient  resources  were 
available  at  the  start  of  time  frame  11  or  12  (the  correct  allocation  was  made,  but  it  was  too 
late),  insufficient  resources  were  available  throughout  time  frames  10,  11  and  12. 
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The  results  are  shown  in  Figure  1 1 .  A  loglinear  model  was  fitted  to  the  data.  The  best-fitting 
model  contained  the  variables  “condition”,  “block”,  “category  of  resource  allocation”,  and 
the  2-way  interactions  between  condition  and  category  and  block  and  category.  Neither  the 
interaction  between  condition  and  block  nor  the  three-way  interaction  between  condition, 
block  and  category  were  significant. 


Fig.  1 1  Performance  measurements  during  time  frame  10. 


There  was  a  significant  interaction  when  comparing  the  individuals  with  the  unrestricted 
communication  teams,  x^(ll)=35.37,  /j<  .001.  Inspection  of  Figure  11  shows  that  this  result 
is  caused  by  the  unrestricted  communication  group  having  allocated  sufficient  resources  in 
more  scenarios  than  the  individuals  (sufficient  resources  were  allocated  in  time  frame  10  by 
29%  of  the  unrestricted  communicating  teams  versus  12%  of  the  individuals).  In  addition,  the 
individuals  were  more  often  too  late  than  the  unrestricted  communication  group  (65%  versus 
40%,  respectively).  The  interaction  was  also  significant  when  comparing  the  unrestricted 
communication  group  and  the  restricted  communication  group  (both  with  shared  knowledge), 
l\\\)=A\A9,  /7<.001.  Inspection  of  Figure  II  shows  that  this  result  is  caused  by  the 
unrestricted  communication  group  having  allocated  sufficient  resources  in  more  scenarios  than 
the  restricted  communication  group  (sufficient  resources  were  allocated  in  time  frame  10  by 
29%  of  the  unrestricted  communicating  teams  versus  7%  of  the  restricted  communicating 
teams).  In  addition,  the  restricted  communication  group  was  more  often  too  late  than  the 
unrestricted  communication  group  (66%  versus  40%,  respectively).  The  interaction  of 
condition  by  resource  allocation  was  significant  when  comparing  the  restricted  communication 
groups  without  and  with  shared  knowledge,  x^(ll)=38.19,  p<.001.  Inspection  of  Figure  11 
shows  that  when  knowledge  was  shared,  more  frequently  insufficient  resources  were  available 
(8%  in  the  restricted  communication  group  without  shared  knowledge  versus  27%  with  shared 
knowledge),  whereas  when  knowledge  was  not  shared,  most  often  the  allocation  was  made 
too  late  (90%  without  shared  knowledge  versus  66%  in  the  restricted  communication  group 
with  shared  knowledge). 
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The  interaction  of  block  by  resource  allocation  was  significant  across  the  four  conditions, 
X2(33)=63.71,  p  =  .001.  Figure  12  shows  that  this  effect  was  largely  caused  by  the  transition 
from  the  first  two  routine  blocks  to  the  last  two  non-routine  blocks.  Compared  with  the 
routine  scenarios,  in  the  non-routine  scenarios  resources  were  more  often  allocated  too  late 
rather  than  in  insufficient  numbers.  This  is  most  likely  due  to  the  fact  that  observers  had  to 
search  longer  for  the  threat  in  the  non-routine  scenarios.  This,  in  turn,  led  to  a  later 
transmission  of  the  threat  to  the  allocator  who  could  only  then  assign  resources  to  the  threat. 


Fig.  12  Interaction  block  by  resource. 


5.4  Timing  of  actions  and  communication  of  messages 

In  some  conditions,  actions  were  not  carried  out  or  messages  were  not  sent  by  all  subjects. 
The  repeated-measures  analysis  of  variance  was  then  carried  out  with  missing  data;  in  those 
cases  where  too  many  data  were  missing  (and  hence  complete  teams  were  omitted  from  the 
analysis),  the  four  scenarios  within  one  block  were  averaged,  resulting  in  four  measures  for 
each  team.  Obviously,  in  the  latter  case  no  effects  of  type  of  scenario  could  be  tested  any 
more.  Where  possible,  both  approaches  to  missing  data  were  taken  simultaneously  and  the 
results  were  cross-checked.  Where  discrepancies  arise,  this  will  be  noted  in  the  text. 

Sending  and  reading  the  last  building  of  the  pattern 

On  average,  the  last  building  of  the  pattern  was  sent  after  105.81  s  by  the  individuals,  106.58 
s  by  the  unrestricted  communication  group,  96.71  s  by  the  restricted  communication  group 
without  shared  knowledge,  and  100.32  s  by  the  communication  group  with  shared  knowledge. 
Given  the  large  number  of  missing  cases  particularly  with  the  individuals,  a  repeated- 
measures  analysis  of  variance  was  carried  out  on  the  block  averages.  The  overall  ANOVA 
across  the  four  conditions  was  not  significant,  F(3,36)  =  1.18,  p  —  .33,  nor  were  any  pairwise 
differences  between  the  conditions  significant.  On  average,  the  last  building  of  the  pattern  was 
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read  after  112.47  s  by  the  individuals,  111.05  s  by  the  unrestricted  communication  group, 
101.97  s  by  the  restricted  communication  group  without  shared  knowledge,  and  104.86  s  by 
the  restricted  communication  group  with  shared  knowledge.  Using  block  averages  because  of 
missing  data,  no  significant  differences  were  found,  all  Fs  <  1 . 

Start  of  search  for  threat  message 

On  average,  the  search  for  the  threat  message  started  after  102.51  s  for  the  individuals,  94.76 
s  for  the  unrestricted  communication  group,  111.72  s  for  the  restricted  communication  group 
without  shared  knowledge,  and  98.77  s  for  the  restricted  communication  with  shared 
knowledge. 

The  overall  difference  among  the  four  conditions  was  significant,  F(3,39)  =  10.56,  /i<.001. 
Planned  comparisons  showed  significant  main  effects  of  condition  for  the  individuals  and  the 
unrestricted  communication  group,  F(l,20)=4.48,  p  =  .05,  for  the  unrestricted  communication 
and  restricted  communication  with  shared  knowledge  groups,  F(l,20)  =  11.02,  p  =  .003,  and 
for  the  restricted  communication  groups  without  and  with  shared  knowledge,  F(l, 19)— 26.85, 
/><  .001.  The  interaction  between  condition  and  type  of  scenario  (routine  versus  non-routine 
scenarios)  was  significant  only  when  comparing  the  restricted  communication  without  and 
with  shared  knowledge  groups,  F(l,19)=5.48,  p  —  .03.  Inspection  of  the  means  indicated  that 
the  restricted  communication  group  without  shared  knowledge  acmally  started  searching 
earlier  for  the  threat  message  in  the  non-routine  scenarios  (routine;  115.39  s;  non-routine: 
108.06  s),  whereas  the  restricted  communication  group  with  shared  knowledge  did  not  differ 
in  both  types  of  scenarios  (routine:  99.01  s;  non-routine:  98.53  s).  This  effect  is  probably 
caused  by  the  initiative  taken  by  the  observers  in  the  restricted  communication  group  without 
shared  knowledge  to  start  searching  for  the  threat  message  in  the  absence  of  any  directions 
given  by  the  allocator.  This  initiative  could  only  be  taken  after  some  experience  was  built  up 
with  searching  for  threat  messages.  Given  that  the  non-routine  scenarios  were  always  the  final 
eight  scenarios,  this  learning  effect  is  inherently  confounded  with  the  effect  of  type  of 
scenario.  No  significant  interactions  between  condition  and  type  of  communication  (normal 
versus  disturbed)  were  found. 

Sending  and  reading  the  threat  message 

On  average,  the  threat  message  was  sent  by  the  observer  after  111.10  s  for  the  individuals, 
107.80  s  for  the  unrestricted  communication  group,  118.92  s  for  the  restricted  communication 
group  without  shared  knowledge,  and  111.44  s  for  the  restricted  communication  group  with 
shared  knowledge.  Planned  comparisons  on  block  averages  revealed  marginally  significant 
differences  between  the  individuals  and  the  unrestricted  communication  group,  F(  1,1 9) =3. 24, 
p  =  .09,  the  unrestricted  communication  group  and  the  restricted  communication  group  with 
shared  knowledge,  F(l,20)=3.72,  p  =  .07and  a  significant  difference  between  the  restricted 
communication  groups  with  and  without  shared  knowledge,  F(l,14)  =  5.41,  p  =  .04.  These  data 
apply  only  to  those  cases  where  a  threat  message  was  in  fact  sent. 
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On  average,  the  threat  message  was  never  sent  in  17%  of  the  cases  for  the  individuals,  15% 
of  the  cases  for  the  unrestricted  communication  group,  54%  of  the  cases  for  the  restricted 
communication  group  without  shared  knowledge,  and  19%  of  the  cases  for  the  restricted 
communication  group  with  shared  knowledge.  The  relatively  high  percentage  of  cases  that  the 
threat  message  was  never  sent  by  the  observers  in  the  restricted  communication  group  without 
shared  knowledge  can  be  explained  by  noting  that  in  distorted  e-mail  scenarios  the  allocator 
never  sees  a  pattern  and  hence  can  never  instruct  the  observer  to  search  for  a  threat  message. 
The  observer  can  only  search  on  his  or  her  own  initiative,  as  a  result  of  previous  learning. 
Fig.  13  presents  the  search  task  for  the  four  conditions  graphically. 
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Fig.  13  Start  and  stop  of  the  search  task  in  the  four  conditions.  The  percentages  denote 
the  deadlines  reached. 


On  average,  the  threat  message,  if  sent,  was  read  by  the  allocator  after  117.12  s  for  the 
individuals,  113.91  s  for  the  unrestricted  communication  group,  120.98  s  for  the  restricted 
communication  group  without  shared  knowledge,  and  115.77  s  for  the  restricted 
communication  group  with  shared  knowledge.  Planned  comparisons  showed  no  significant 
differences  among  these  groups  (individuals  versus  unrestricted  communication  group: 
F(l,20)  =  1.08,  p  =  .'i\\  unrestricted  communication  group  versus  restricted  communication 
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group  with  shared  knowledge:  F(1,20)<1;  restricted  communication  groups  without  versus 
with  shared  knowledge:  F(  1,1 5)  =  1.00,  p  =  .33). 


5.5  Observer  rating 

The  communication  which  took  place  during  the  unrestricted  communication  condition  was 
rated  into  the  different  categories  as  described  in  §  3.5.  Table  II  gives  an  overview  of  the 
mean  number  of  utterances  per  scenario  for  each  category  (the  means  are  also  divided  in 
utterances  during  a  scenario  and  between  scenarios  when  subjects  waited  for  the  next  scenario 
to  start). 


Table  II  Mean  number  of  utterances  per  category  during  a  scenario,  between  a 
scenario  and  total. 
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0.58 
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0.93 
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1.14 

task  division 
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0.29 
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0.09 

0.35 

task  overtake 

1.26 
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1.64 

task  communication  control 
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total  domain  knowledge  communication 
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0.45 
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0.43 
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1.24 

meta  communication 

0.39 

0.70 

0.14 

0.61 

0.53 

0.99 

remaining  communication 

0.77 

1.09 

0.30 

0.68 

1.07  l;40 

The  following  conclusions  can  be  drawn  from  table  5.1.  First,  communication  took  place  both 
during  and  between  scenarios  but  much  more  during  than  in  between.  Second,  data  communi¬ 
cation,  which  only  took  place  during  scenarios,  took  the  largest  part  of  the  total 
communication.  Apparently,  while  data  communication  could  perfectly  take  place  via 
standardized  e-mail,  subjects  felt  the  need  to  communicate  this  information  directly  to  each 
other.  Note  that  subjects  gave  each  other  more  information  than  explicitly  asked  for  (e.g., 
more  answers  were  given  then  questions  asked).  Third,  during  scenarios,  subjects 
communicated  a  great  deal  about  the  task  itself.  Most  of  this  communication  was  logging. 
Subjects  tended  to  tell  each  other  continuously  about  the  activities  they  were  undertaking. 
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Fourth,  during  as  well  between  scenarios,  subjects  were  communicating  about  past  actions 
and  the  strategies  used  in  order  to  improve  performance  (domain  knowledge  communication). 

Figures  14- 16  give  an  overview  of  the  relative  contribution  of  each  communication  category 
to  the  total  (including  during  and  between  scenarios  as  well  as  both). 


Fig.  14  Relative  contribution  of  each  communication  category  to  the  total  during 
scenarios  as  well  as  between  scenarios. 


Fig.  15  Relative  contribution  of  each  communication  category  to  the  total  during 
scenarios. 
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Fig.  16  Relative  contribution  of  each  communication  category  to  the  total  between 
scenarios. 
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During  the  tasks,  half  of  the  information  was  about  data  needed  for  task  execution. 
Approximately  a  third  (34%)  of  the  communication  during  tasks  was  about  performing  the 
task  (task  communication).  Approximately  a  tenth  (11%)  of  the  communication  was 
exchanging  domain  knowledge.  The  amount  of  meta-communication  was  small  (2%),  and  so 
was  the  remaining  communication  (3%).  Between  the  scenarios,  the  distribution  of 
communication  types  was  very  different.  Obviously,  no  data  communication  took  place.  Two- 
thirds  of  the  communication  was  domain  knowledge  communication  (66%),  and  one  fifth  was 
on  performing  the  tasks  (18%).  Some  remaining  communication  took  place  in  11%  of  the 
utterances,  and  the  amount  of  meta-communication  was,  again,  small  (5%). 


6  DISCUSSION 

In  this  report,  we  started  with  describing  a  framework  for  knowledge  and  information  use 
within  teams,  and  continued  with  focusing  on  the  necessity  for  working  with  more  than  one 
agent,  the  conditions  of  communication  and  coordination,  and  the  distribution  of  knowledge 
within  a  team.  The  hypotheses,  based  on  the  framework  and  an  extensive  analysis  of  a 
command  &  control  task,  were  tested  experunentally .  In  this  section  we  discuss  the 
experimental  results  and  how  to  continue  this  type  of  research. 


6.1  Experimental  results 

Three  comparisons  were  made  in  order  to  test  the  hypotheses  stated  in  §  3.4.  The  first 
comparison  was  between  individuals  and  teams  that  had  unrestricted  communication 
possibilities.  The  results  show  that  forming  a  team  of  two  agents  (instead  of  doing  the  job 
alone)  benefits  the  overall  performance  of  the  fire-fighting  task.  During  the  critical  time 
period,  teams  had  more  fire-fighting  units  available  than  individuals.  Consequently,  more 
often  sufficient  units  were  allocated  in  the  teams  condition  compared  to  the  individual 
condition.  Also,  individuals  were  more  often  too  late.  The  analysis  of  the  fire-fighting  task 
had  already  made  clear  that  for  effective  execution,  certain  activities  had  to  be  performed  in 
parallel.  That  individuals  could  not  perform  certain  activities  in  parallel  is  also  made  clear  by 
the  fact  that  individuals  were  later  than  teams  in  searching  for  and  sending  the  threat  message. 
Since  the  results  of  the  experiment  support  the  conclusion  that  certain  tasks  had  to  be 
performed  in  parallel,  we  may  state  that  the  fire-fighting  task  is  truly  a  team  task. 

The  second  comparison  was  made  between  teams  that  had  unrestricted  communication 
possibilities  and  teams  that  were  restricted  in  their  communication  to  data  communication 
only.  The  a  priori  analysis  of  the  fire-fighting  task  had  already  made  clear  that  restricting 
communication  to  data  does  not  necessarily  lead  to  a  performance  decrease.  In  other  words, 
data  communication  (via  standardized  e-mail)  provides  the  same  possibilities  for  effective  task 
execution  as  unrestricted  communication.  However,  the  results  show  that  teams  with 
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unrestricted  communication  possibilities  perform  better  than  teams  that  have  restricted 
communication  possibilities.  During  the  critical  time  period,  unrestricted  communication 
teams  had  more  resources  available  than  the  restricted  communication  teams,  and  more  often 
sufficient  units  were  allocated.  In  addition,  compared  with  the  unrestricted  communication 
teams,  the  restricted  communication  teams  were  more  often  too  late  with  allocating  units. 
Finally,  the  unrestricted  communication  group  began  earlier  with  searching  for  the  threatened 
building  than  the  restricted  communication  group,  and  they  were  earlier  with  sending  a 
message  about  the  threatened  building,  once  found. 

Apparently,  restricting  communication  to  data  only  leads  to  a  performance  decrease.  Two 
explanations  can  be  put  forward  for  this  result.  A  first  explanation  is  that  teams  with 
unrestricted  communication  possibilities  had  more  opportunities  for  coordination.  The  results 
of  the  observer  rating  seem  to  support  this  explanation.  Besides  data  communication,  most 
communication  took  place  about  the  task.  Subjects  communicated  a  great  deal  about  their  task 
execution,  and  informed  each  other  especially  what  activities  they  were  doing  on  a  particular 
moment  (i.e.,  logging).  This  increased  insight  in  the  task  performance  of  others,  and  in  the 
consequences  of  one’s  own  task  performance  for  that  of  others,  may  have  provided  a  better 
coordination  in  performing  tasks. 

An  alternative  explanation  for  the  results  is  that  in  teams  with  unrestricted  communication, 
more  learning  could  have  taken  place  to  optimize  the  task.  The  need  for  optimization  is 
indicated  by  the  overall  low  performance  in  all  conditions.  Objectively,  the  time  for  reflection 
about  the  task,  usually  taking  place  at  quiet  moments  after  the  critical  period  and  between  the 
scenarios,  was  equal  in  both  conditions.  However,  it  is  known  from  the  literature  on 
cooperative  learning  (see  e.g.,  Druckman  &  Bjork,  1994)  that  working  together  can  be 
beneficial  for  co-workers.  Our  own  results,  showing  that  teams  perform  better  than 
individuals,  are 'consistent  with  a  cooperative  learning  effect  as  an  explanation.  Further,  when 
we  look  at  the  observational  data  in  detail,  the  second  most  communication  between  the  team 
members  (besides  data  communication),  after  task  communication,  is  about  domain 
knowledge.  This  type  of  communication  took  place  during  as  well  as  between  scenarios, 
indicating  that  teams  continuously  monitor  their  previously  used  strategies,  discuss  and  adjust 
them  when  necessary.  This  behaviour,  which  is  also  associated  with  the  team  self-correction 
phenomenon  as  described  by  Blickensderfer,  Cannon-Bowers  and  Salas  (in  press),  was  absent 
in  the  restricted  communication  condition. 

The  third  comparison  was  made  between  teams  with  restricted  communication  possibilities 
without  and  with  shared  knowledge.  Sharing  knowledge  gives  team  members  the  possibility  to 
act  more  independently  of  each  other  and  substitute  for  each  other’s  tasks  when  necessary. 
The  task  analysis  shows  that  this  can  be  profitable  in  the  fire-fighting  task.  Therefore,  it  is 
assumed  that  sharing  knowledge  across  team  members  contributes  to  the  overall  team 
performance.  The  results  of  the  experiment  support  this  assumption.  Although  both  restricted 
communication  teams  had  insufficient  resources  available  as  well  as  allocated,  the  teams  with 
shared  knowledge  more  often  allocated  insufficient  resources,  whereas  the  teams  without 
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shared  knowledge  were  too  late  with  the  allocation  of  resources  in  almost  all  cases.  This  result 
indicates  that  teams  with  shared  knowledge  were  faster  in  allocating  resources  than  teams 
without  shared  knowledge.  Besides  that,  the  shared  knowledge  group  was  also  earlier  in 
searching  the  threat  message  and  sending  the  threat  message.  On  basis  of  these  results  we  may 
conclude  that  shared  knowledge  contributed  to  an  increased  performance  of  the  restricted 
communication  teams. 

Summarized,  the  following  three  main  conclusions  can  be  drawn  from  the  experimental 
results.  First,  the  fire-fighting  task  truly  is  a  team  task  because  for  optimal  task  performance, 
crucial  task  elements  should  be  carried  out  in  parallel.  Second,  communication  restricted  to 
data  exchange  only  leads  to  team  performance  decrease.  Unclear  is  whether  this  is  caused  by 
absent  communication  of  domain  knowledge  needed  for  cooperative  learning  or  team  self 
correction,  or  due  to  absent  communication  of  task  information  needed  for  coordination. 
Third,  distributing  knowledge  over  team  members  leads  to  an  increase  of  parallel  task 
execution  and  to  better  team  performance. 


6.2  Further  research 

We  do  not  know  yet  which  mechanisms  underlies  the  beneficial  effects  of  unrestricted 
communication:  learning,  communicating  coordination  information,  or  both.  These  two 
factors  will  be  the  topic  of  our  future  research.  There  has  been  done  already  research  on  this 
matter  (e.g.,  Fussell  &  Benimoff,  1995),  but  in  contrast  to  others  that  focus  on  a 
communication  perspective,  our  research  is  carried  out  from  a  task  perspective.  In  the  next 
experiment,  also  taking  place  in  the  fire-fighting  paradigm,  team  members  will  be  separated 
in  order  to  prevent  cooperative  learning,  and  communication  will  be  restricted.  In  some  of  the 
separated  teams,  coordination  related  information  will  be  communicated  too  by  means  of  a 
dynamic  visualization  of  the  task  dependencies  between  the  agents.  It  is  hypothesized  that 
exchanging  coordination  information  improves  team  performance.  In  theory,  the  experiment 
could  be  done  with  teams  communicating  without  restrictions,  but  in  practice,  it  will  be 
difficult  to  prevent  subjects  from  communicating  information  for  learning  during  task 
execution. 

The  results  of  our  experiments  in  the  fire-fighting  paradigm  seem  generalizable  to  teams 
performing  other  tasks,  such  as  command  &  control.  The  fire-fighting  task  has  been 
decomposed  into  four  tasks:  Situation  Awareness,  Threat  Assessment,  Decision  Making/ 
Planning,  and  Direction  &  Control.  The  same  decomposition  can  be  used  for  other  team 
tasks,  such  as  Command  &  Control.  As  was  shown  in  Figure  5,  both  the  tasks  and  the  types 
of  knowledge  are  formulated  in  a  generic  way  (i.e.,  when  Fire  Data  is  replaced  by  Data). 
However,  in  command  &  control,  the  tasks  are  usually  divided  over  more  than  one  domain. 
Because  different  domain  experts  can  hold  different  viewpoints  of  a  situation,  conflicts 
between  the  domains  can  arise  within  the  same  function.  For  improving  generalizability 
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further,  the  paradigm  should  enable  the  occurrence  of  domain  conflicts.  Then,  command  & 
control  teams  solving  domain  conflicts  can  be  studied  as  well. 

Another  restriction  of  the  fire-fighting  domain,  as  it  has  been  realized  in  the  experiments,  is 
the  level  of  uncertainty  in  the  information  used  for  performing  the  tasks.  Two  types  of 
uncertainty  can  be  distinguished;  uncertainty  of  situational  data  (in  the  situation  assessment 
task),  and  uncertainty  on  the  patterns  that  indicate  a  threat  (in  the  threat  assessment  task).  In 
contrast  to  real-world  settings,  the  level  of  uncertainty  in  the  current  paradigm  was  low. 
Further  research  may  involve  experiments  with  higher  levels  of  uncertainty  of  both  kinds.  An 
associated  aspect  is  the  distinction  between  handling  routine  and  non-routine  situations.  In  the 
experiments,  the  non-routine  scenarios  we  used  could  be  solved  with  routine  behaviour.  The 
only  difference  was  that  the  surveillance  task,  consisting  of  searching,  was  carried  out  over  a 
longer  period  of  time.  Real  non-routine  scenarios  demand  knowledge-based  problem  solving. 
How  teams  will  handle  these  kinds  of  scenarios  needs  to  be  investigated  as  well  (cf.  Cohen, 
1993). 

Further,  we  carried  out  the  experiment  with  two  types  of  team  organization:  a  single  agent 
and  a  team.  Several  other  types  of  team  organization  could  be  the  object  of  research.  The 
organization  of  a  team  can  be  based  on  domains,  on  problem  solving  expertise  (e.g.,  a 
division  in  scepticism,  critics,  and  proposers;  see  Kornfeld  &  Hewitt,  1981),  on  function,  on 
control  role  (supervision,  execution,  etc.).  So,  for  a  particular  team  size  (say  3  members), 
different  team  organizations  may  be  compared  that  deal  with  exactly  the  same  environmental 
events.  The  role  of  various  aspects  can  be  investigated  in  this  way.  For  example,  in  some 
forms  of  organization,  face-to-face  contact  may  be  essential,  while  in  others  this  may  not  be 
the  case  (especially  where  no  negotiation  takes  place).  See,  for  example,  Williams  (1977); 
Werkhoven,  Punte  and  Spoelma  (in  press).  Another  question  that  may  be  investigated  is  the 
optimal  size  and  structure  of  a  team.  Subquestions  are;  assuming  that  team  members  need  a 
mental  model  of  each  colleague,  how  many  models  can  they  handle?  To  what  extent  does  this 
depend  on  the  differences  in  roles  of  the  team  member? 

A  final  topic  for  further  research  is  how  to  measure  team  performance.  Team  performance  is 
not  measured  easily.  A  difficulty  is  that  much  team  behaviour  is  covert  and  implicit  while 
only  overt  and  explicit  behaviour  can  be  measured.  Common  performance  measures  are 
correct  task  completion,  and  the  amount  of  time  consumed.  Two  problems  exist  with 
measuring  correct  task  completion.  First,  correct  task  completion  is  often  difficult  to 
determine  due  to  a  lack  of  a  gold  standard  definition.  Second,  often  the  task  is  made  up  of 
several  sub-tasks  and  the  result  of  some  tasks  may  depend  on  the  outcome  of  others.  These 
sub-tasks  can  be  rated  as  well.  Consider  a  soccer-team:  a  coordinated  action  with  two  players 
to  pass  by  an  opponent  may  be  appreciated  equally  well  by  the  supporters  as  perhaps  a  lucky 
goal  that  may  follow.  In  other  words,  enabling  the  conditions  for  success  can  be  rated  as  well, 
especially  when  the  result  depends  partially  on  chance.  Further,  not  only  task  performance 
itself  but  also  the  gain  of  a  correct  task  performance  can  be  taken  into  account.  In  addition, 
the  errors  that  are  made  during  (sub)task  performance  (and  the  associated  costs)  can  be  taken 
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as  a  measure.  Rouse  et  al.  (1992)  use  an  analysis  method  for  human  errors.  Errors  with  clear 
consequences  are  important,  but  also  errors  that  increase  the  probability  of  serious 
consequences,  even  when  the  final  result  may  be  positive. 

The  second  common  performance  measure,  time  consumption,  can  be  measured  in  terms  of 
time  taken  to  carry  out  a  job,  but  also  in  terms  of  reaching  deadlines.  Especially  in  teams,  in 
which  the  teams  wait  for  a  critical  situation  to  occur,  performing  a  Job  as  quickly  as  possible 
is  not  important,  as  long  as  they  can  perform  a  Job  under  critical  circumstances  within  time; 
they  cannot  increase  their  productivity  by  speeding  up  their  tasks  (except  that  more  time  may 
become  available  for  task  evaluation,  training,  and  relaxation.) 

In  short,  what  is  needed  for  research  on  team  decision  making  is  an  instrument  for  measuring 
shared  capacity  optimization. 


6.3  Conclusions 

In  conclusion,  we  have  contributed  to  the  understanding  of  how  to  distinguish  an  expert  team 
from  a  team  of  experts.  We  have  found  that  well-performing  teams  do  more  than  working 
separately,  only  communicating  the  results  of  their  actions.  Teams  also  need  to  communicate 
additional  information,  probably  information  such  as  knowledge  they  own  or  have  learned 
and,  in  order  to  enhance  coordination,  information  about  what  they  are  doing.  Especially  in 
our  current  society,  in  which  more  and  more  often  people  are  enabled  or  forced  to  work 
together  but  distributed  in  time  and  place,  the  role  of  communication  and  coordination 
deserves  considerable  attention. 
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